2024-04-25T11:05:16.925 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T11:05:16.926 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120 --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-readahead} -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/orig.config.yaml 2024-04-25T11:05:16.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:05:17.057 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-readahead} email: pdonnell@redhat.com first_in_suite: false job_id: '7673120' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDApDp15FJhIISdM89JUbqpyW3/JA5lfpUD9DI3r/nygW1U9UCccNZS89Rdm6RlS06064mYoyJt6pyNKYcMMJNI= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH5LKZnKvuNKEJw8Zd9Hhfl9/xjgNBUgjsKyzzMjcJ4WOEBQ6PsWuqxm/RM5rbMWFwprEEuGh2HBCkQnY3CCJZM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_readahead teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T11:05:17.057 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T11:05:17.058 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T11:05:17.058 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T11:05:17.060 INFO:teuthology.task.internal:Checking packages... 2024-04-25T11:05:17.083 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T11:05:17.083 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T11:05:17.083 INFO:teuthology.packaging:ref: None 2024-04-25T11:05:17.083 INFO:teuthology.packaging:tag: None 2024-04-25T11:05:17.083 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T11:05:17.084 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:05:17.084 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T11:05:17.230 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:05:17.232 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T11:05:17.239 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T11:05:17.239 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T11:05:17.256 INFO:teuthology.task.internal:Saving configuration 2024-04-25T11:05:17.272 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T11:05:17.279 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:05:17.303 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:59:41.724979', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGC+yI++cD4v4kQKbHlRTqO7RlgEwvhBcLCwuyA7S61M'} 2024-04-25T11:05:17.324 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:59:41.723737', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPczCvpEg0qys0cvsj+ovI8/VcuIqhUih114LLMlgyFh'} 2024-04-25T11:05:17.325 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T11:05:17.331 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T11:05:17.331 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T11:05:17.331 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T11:05:17.400 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1cc07243a0>, signals=[15]) 2024-04-25T11:05:17.401 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T11:05:17.407 INFO:teuthology.task.internal:Opening connections... 2024-04-25T11:05:17.407 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:05:17.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:05:17.489 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:05:17.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:05:17.569 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T11:05:17.575 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-04-25T11:05:17.581 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-04-25T11:05:17.581 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Ubuntu" 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="22.04" 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_CODENAME=jammy 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:ID=ubuntu 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE=debian 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T11:05:17.629 INFO:teuthology.orchestra.run.smithi106.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T11:05:17.630 INFO:teuthology.orchestra.run.smithi106.stdout:UBUNTU_CODENAME=jammy 2024-04-25T11:05:17.630 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-04-25T11:05:17.656 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-04-25T11:05:17.661 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-04-25T11:05:17.662 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-04-25T11:05:17.707 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T11:05:17.707 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Ubuntu" 2024-04-25T11:05:17.707 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="22.04" 2024-04-25T11:05:17.707 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T11:05:17.707 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_CODENAME=jammy 2024-04-25T11:05:17.707 INFO:teuthology.orchestra.run.smithi134.stdout:ID=ubuntu 2024-04-25T11:05:17.707 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE=debian 2024-04-25T11:05:17.707 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T11:05:17.708 INFO:teuthology.orchestra.run.smithi134.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T11:05:17.708 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T11:05:17.708 INFO:teuthology.orchestra.run.smithi134.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T11:05:17.708 INFO:teuthology.orchestra.run.smithi134.stdout:UBUNTU_CODENAME=jammy 2024-04-25T11:05:17.708 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-04-25T11:05:17.731 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T11:05:17.740 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T11:05:17.747 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T11:05:17.747 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:05:17.750 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:05:17.754 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T11:05:17.760 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T11:05:17.760 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:05:17.798 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:05:17.807 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T11:05:17.864 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T11:05:17.884 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T11:05:17.884 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T11:05:17.884 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:05:17.885 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:05:17.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:05:17.890 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-25T11:05:17.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:05:17.891 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-04-25T11:05:17.937 INFO:teuthology.orchestra.run.smithi106.stdout:5.15.0-56-generic 2024-04-25T11:05:17.937 INFO:teuthology.task.kernel:Running kernel on smithi106: 5.15.0-56-generic 2024-04-25T11:05:17.938 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get clean 2024-04-25T11:05:17.939 INFO:teuthology.orchestra.run.smithi134.stdout:5.15.0-56-generic 2024-04-25T11:05:17.940 INFO:teuthology.task.kernel:Running kernel on smithi134: 5.15.0-56-generic 2024-04-25T11:05:17.940 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get clean 2024-04-25T11:05:18.073 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-04-25T11:05:18.237 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T11:05:18.237 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T11:05:18.238 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T11:05:18.245 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get update 2024-04-25T11:05:18.325 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T11:05:18.398 INFO:teuthology.orchestra.run.smithi106.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T11:05:18.493 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T11:05:18.503 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T11:05:18.566 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T11:05:18.576 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T11:05:18.587 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T11:05:18.616 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T11:05:18.620 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T11:05:18.623 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T11:05:18.624 INFO:teuthology.orchestra.run.smithi134.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T11:05:18.624 INFO:teuthology.orchestra.run.smithi134.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T11:05:18.653 INFO:teuthology.orchestra.run.smithi134.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T11:05:18.657 INFO:teuthology.orchestra.run.smithi134.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T11:05:18.657 INFO:teuthology.orchestra.run.smithi134.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T11:05:18.666 INFO:teuthology.orchestra.run.smithi134.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T11:05:18.673 INFO:teuthology.orchestra.run.smithi134.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T11:05:18.675 INFO:teuthology.orchestra.run.smithi134.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T11:05:18.692 INFO:teuthology.orchestra.run.smithi134.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T11:05:18.692 INFO:teuthology.orchestra.run.smithi134.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T11:05:18.692 INFO:teuthology.orchestra.run.smithi134.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T11:05:18.693 INFO:teuthology.orchestra.run.smithi134.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T11:05:18.697 INFO:teuthology.orchestra.run.smithi134.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T11:05:18.699 INFO:teuthology.orchestra.run.smithi134.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T11:05:18.700 INFO:teuthology.orchestra.run.smithi134.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T11:05:18.702 INFO:teuthology.orchestra.run.smithi134.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T11:05:18.720 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T11:05:18.725 INFO:teuthology.orchestra.run.smithi134.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T11:05:18.726 INFO:teuthology.orchestra.run.smithi134.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T11:05:18.730 INFO:teuthology.orchestra.run.smithi134.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T11:05:18.731 INFO:teuthology.orchestra.run.smithi134.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T11:05:18.743 INFO:teuthology.orchestra.run.smithi134.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T11:05:18.750 INFO:teuthology.orchestra.run.smithi134.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T11:05:18.752 INFO:teuthology.orchestra.run.smithi134.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T11:05:18.753 INFO:teuthology.orchestra.run.smithi134.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T11:05:18.754 INFO:teuthology.orchestra.run.smithi134.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T11:05:18.773 INFO:teuthology.orchestra.run.smithi134.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T11:05:18.774 INFO:teuthology.orchestra.run.smithi134.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T11:05:18.774 INFO:teuthology.orchestra.run.smithi134.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T11:05:18.774 INFO:teuthology.orchestra.run.smithi134.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T11:05:18.776 INFO:teuthology.orchestra.run.smithi134.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T11:05:18.776 INFO:teuthology.orchestra.run.smithi134.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T11:05:18.776 INFO:teuthology.orchestra.run.smithi134.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T11:05:18.776 INFO:teuthology.orchestra.run.smithi134.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T11:05:18.782 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T11:05:18.787 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T11:05:18.788 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T11:05:18.789 INFO:teuthology.orchestra.run.smithi106.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T11:05:18.796 INFO:teuthology.orchestra.run.smithi134.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T11:05:18.796 INFO:teuthology.orchestra.run.smithi134.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T11:05:18.818 INFO:teuthology.orchestra.run.smithi106.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T11:05:18.822 INFO:teuthology.orchestra.run.smithi106.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T11:05:18.822 INFO:teuthology.orchestra.run.smithi106.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T11:05:18.832 INFO:teuthology.orchestra.run.smithi106.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T11:05:18.838 INFO:teuthology.orchestra.run.smithi106.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T11:05:18.840 INFO:teuthology.orchestra.run.smithi106.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T11:05:18.840 INFO:teuthology.orchestra.run.smithi106.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T11:05:18.840 INFO:teuthology.orchestra.run.smithi106.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T11:05:18.859 INFO:teuthology.orchestra.run.smithi106.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T11:05:18.859 INFO:teuthology.orchestra.run.smithi106.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T11:05:18.948 INFO:teuthology.orchestra.run.smithi106.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T11:05:19.086 INFO:teuthology.orchestra.run.smithi106.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T11:05:19.395 INFO:teuthology.orchestra.run.smithi106.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T11:05:19.434 INFO:teuthology.orchestra.run.smithi106.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T11:05:19.450 INFO:teuthology.orchestra.run.smithi106.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T11:05:19.451 INFO:teuthology.orchestra.run.smithi106.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T11:05:19.452 INFO:teuthology.orchestra.run.smithi106.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T11:05:19.513 INFO:teuthology.orchestra.run.smithi106.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T11:05:19.524 INFO:teuthology.orchestra.run.smithi106.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T11:05:19.688 INFO:teuthology.orchestra.run.smithi106.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T11:05:20.326 INFO:teuthology.orchestra.run.smithi106.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T11:05:20.364 INFO:teuthology.orchestra.run.smithi106.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T11:05:20.377 INFO:teuthology.orchestra.run.smithi106.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T11:05:20.378 INFO:teuthology.orchestra.run.smithi106.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T11:05:20.378 INFO:teuthology.orchestra.run.smithi106.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T11:05:20.380 INFO:teuthology.orchestra.run.smithi106.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T11:05:20.381 INFO:teuthology.orchestra.run.smithi106.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T11:05:20.381 INFO:teuthology.orchestra.run.smithi106.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T11:05:20.409 INFO:teuthology.orchestra.run.smithi106.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T11:05:20.413 INFO:teuthology.orchestra.run.smithi106.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T11:05:20.578 INFO:teuthology.orchestra.run.smithi106.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T11:05:20.660 INFO:teuthology.orchestra.run.smithi106.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T11:05:20.784 INFO:teuthology.orchestra.run.smithi106.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T11:05:20.832 INFO:teuthology.orchestra.run.smithi106.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T11:05:20.863 INFO:teuthology.orchestra.run.smithi106.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T11:05:24.215 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 13.3 MB in 2s (7,083 kB/s) 2024-04-25T11:05:25.140 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 13.3 MB in 3s (5,222 kB/s) 2024-04-25T11:05:25.681 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-04-25T11:05:25.698 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T11:05:25.758 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-04-25T11:05:25.947 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-04-25T11:05:25.948 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-04-25T11:05:26.107 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T11:05:26.107 INFO:teuthology.orchestra.run.smithi134.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T11:05:26.107 INFO:teuthology.orchestra.run.smithi134.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T11:05:26.107 INFO:teuthology.orchestra.run.smithi134.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T11:05:26.107 INFO:teuthology.orchestra.run.smithi134.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T11:05:26.108 INFO:teuthology.orchestra.run.smithi134.stdout: python2.7-minimal 2024-04-25T11:05:26.108 INFO:teuthology.orchestra.run.smithi134.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T11:05:26.109 INFO:teuthology.orchestra.run.smithi134.stdout:The following additional packages will be installed: 2024-04-25T11:05:26.109 INFO:teuthology.orchestra.run.smithi134.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T11:05:26.109 INFO:teuthology.orchestra.run.smithi134.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T11:05:26.110 INFO:teuthology.orchestra.run.smithi134.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T11:05:26.111 INFO:teuthology.orchestra.run.smithi134.stdout:Suggested packages: 2024-04-25T11:05:26.111 INFO:teuthology.orchestra.run.smithi134.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T11:05:26.111 INFO:teuthology.orchestra.run.smithi134.stdout:Recommended packages: 2024-04-25T11:05:26.111 INFO:teuthology.orchestra.run.smithi134.stdout: thermald 2024-04-25T11:05:26.164 INFO:teuthology.orchestra.run.smithi134.stdout:The following NEW packages will be installed: 2024-04-25T11:05:26.164 INFO:teuthology.orchestra.run.smithi134.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T11:05:26.164 INFO:teuthology.orchestra.run.smithi134.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T11:05:26.164 INFO:teuthology.orchestra.run.smithi134.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T11:05:26.166 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages will be upgraded: 2024-04-25T11:05:26.166 INFO:teuthology.orchestra.run.smithi134.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi134.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi134.stdout:Need to get 113 MB of archives. 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi134.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi134.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T11:05:26.414 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2024-04-25T11:05:26.431 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T11:05:26.493 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2024-04-25T11:05:26.688 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T11:05:26.693 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2024-04-25T11:05:26.694 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2024-04-25T11:05:26.789 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T11:05:26.859 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T11:05:26.859 INFO:teuthology.orchestra.run.smithi106.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T11:05:26.859 INFO:teuthology.orchestra.run.smithi106.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T11:05:26.859 INFO:teuthology.orchestra.run.smithi106.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T11:05:26.860 INFO:teuthology.orchestra.run.smithi106.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T11:05:26.860 INFO:teuthology.orchestra.run.smithi106.stdout: python2.7-minimal 2024-04-25T11:05:26.860 INFO:teuthology.orchestra.run.smithi106.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T11:05:26.860 INFO:teuthology.orchestra.run.smithi106.stdout:The following additional packages will be installed: 2024-04-25T11:05:26.860 INFO:teuthology.orchestra.run.smithi106.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T11:05:26.860 INFO:teuthology.orchestra.run.smithi106.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T11:05:26.861 INFO:teuthology.orchestra.run.smithi106.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T11:05:26.862 INFO:teuthology.orchestra.run.smithi106.stdout:Suggested packages: 2024-04-25T11:05:26.862 INFO:teuthology.orchestra.run.smithi106.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T11:05:26.862 INFO:teuthology.orchestra.run.smithi106.stdout:Recommended packages: 2024-04-25T11:05:26.862 INFO:teuthology.orchestra.run.smithi106.stdout: thermald 2024-04-25T11:05:26.902 INFO:teuthology.orchestra.run.smithi106.stdout:The following NEW packages will be installed: 2024-04-25T11:05:26.902 INFO:teuthology.orchestra.run.smithi106.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T11:05:26.902 INFO:teuthology.orchestra.run.smithi106.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T11:05:26.903 INFO:teuthology.orchestra.run.smithi106.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T11:05:26.904 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages will be upgraded: 2024-04-25T11:05:26.905 INFO:teuthology.orchestra.run.smithi106.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T11:05:27.086 INFO:teuthology.orchestra.run.smithi106.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T11:05:27.087 INFO:teuthology.orchestra.run.smithi106.stdout:Need to get 113 MB of archives. 2024-04-25T11:05:27.087 INFO:teuthology.orchestra.run.smithi106.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T11:05:27.087 INFO:teuthology.orchestra.run.smithi106.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T11:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T11:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T11:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T11:05:28.096 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T11:05:28.138 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T11:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 113 MB in 2s (57.6 MB/s) 2024-04-25T11:05:28.675 INFO:teuthology.orchestra.run.smithi106.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T11:05:28.730 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T11:05:29.020 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T11:05:30.673 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T11:05:30.679 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T11:05:30.801 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:31.153 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T11:05:31.153 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T11:05:31.153 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T11:05:31.717 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T11:05:31.856 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T11:05:32.249 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 113 MB in 5s (22.9 MB/s) 2024-04-25T11:05:32.404 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T11:05:33.401 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T11:05:33.414 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T11:05:33.488 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:33.996 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T11:05:34.017 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T11:05:34.054 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:34.404 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T11:05:34.408 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T11:05:34.467 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:36.643 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T11:05:36.664 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T11:05:36.745 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:37.188 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T11:05:37.197 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T11:05:37.238 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:40.947 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T11:05:41.097 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T11:05:41.472 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T11:05:41.600 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T11:05:41.911 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T11:05:41.934 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T11:05:41.969 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T11:05:43.696 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T11:05:43.821 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T11:05:44.204 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T11:05:44.316 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T11:05:44.577 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T11:05:44.599 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T11:05:44.644 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T11:05:47.876 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T11:05:47.913 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T11:05:47.976 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:50.397 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T11:05:50.433 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T11:05:50.471 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:50.867 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T11:05:50.974 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T11:05:51.346 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T11:05:51.485 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:51.611 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T11:05:51.761 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:53.008 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T11:05:53.100 INFO:teuthology.orchestra.run.smithi134.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T11:05:53.100 INFO:teuthology.orchestra.run.smithi134.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T11:05:53.100 INFO:teuthology.orchestra.run.smithi134.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T11:05:53.100 INFO:teuthology.orchestra.run.smithi134.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T11:05:53.132 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T11:05:53.242 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:53.465 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T11:05:53.978 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:54.480 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T11:05:54.608 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:54.811 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:55.929 INFO:teuthology.orchestra.run.smithi106.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T11:05:55.929 INFO:teuthology.orchestra.run.smithi106.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T11:05:55.929 INFO:teuthology.orchestra.run.smithi106.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T11:05:55.929 INFO:teuthology.orchestra.run.smithi106.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T11:05:56.084 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:56.228 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T11:05:56.371 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T11:05:56.489 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:56.548 INFO:teuthology.orchestra.run.smithi134.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T11:05:56.548 INFO:teuthology.orchestra.run.smithi134.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T11:05:57.495 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:58.871 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T11:05:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T11:05:59.123 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T11:05:59.182 INFO:teuthology.orchestra.run.smithi106.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T11:05:59.182 INFO:teuthology.orchestra.run.smithi106.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T11:06:11.049 INFO:teuthology.orchestra.run.smithi134.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T11:06:11.049 INFO:teuthology.orchestra.run.smithi134.stdout:Sourcing file `/etc/default/grub' 2024-04-25T11:06:11.066 INFO:teuthology.orchestra.run.smithi134.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T11:06:11.129 INFO:teuthology.orchestra.run.smithi134.stdout:Generating grub configuration file ... 2024-04-25T11:06:11.486 INFO:teuthology.orchestra.run.smithi134.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T11:06:11.506 INFO:teuthology.orchestra.run.smithi134.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T11:06:11.934 INFO:teuthology.orchestra.run.smithi134.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T11:06:11.940 INFO:teuthology.orchestra.run.smithi134.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T11:06:12.009 INFO:teuthology.orchestra.run.smithi134.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T11:06:12.014 INFO:teuthology.orchestra.run.smithi134.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T11:06:12.162 INFO:teuthology.orchestra.run.smithi134.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T11:06:12.163 INFO:teuthology.orchestra.run.smithi134.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T11:06:12.163 INFO:teuthology.orchestra.run.smithi134.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T11:06:12.204 INFO:teuthology.orchestra.run.smithi134.stdout:done 2024-04-25T11:06:12.717 DEBUG:teuthology.orchestra.run.smithi134:> dpkg -s linux-image-generic 2024-04-25T11:06:12.752 INFO:teuthology.orchestra.run.smithi134.stdout:Package: linux-image-generic 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Status: install ok installed 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Priority: optional 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Section: kernel 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Installed-Size: 21 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: amd64 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Source: linux-meta 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Version: 5.15.0.105.102 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Recommends: thermald 2024-04-25T11:06:12.753 INFO:teuthology.orchestra.run.smithi134.stdout:Description: Generic Linux kernel image 2024-04-25T11:06:12.754 INFO:teuthology.orchestra.run.smithi134.stdout: This package will always depend on the latest generic kernel image 2024-04-25T11:06:12.754 INFO:teuthology.orchestra.run.smithi134.stdout: available. 2024-04-25T11:06:12.754 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T11:06:12.754 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T11:06:12.754 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T11:06:12.754 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T11:06:12.754 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T11:06:12.755 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get clean 2024-04-25T11:06:12.941 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-04-25T11:06:13.068 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T11:06:13.068 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T11:06:13.080 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T11:06:13.100 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T11:06:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T11:06:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:Sourcing file `/etc/default/grub' 2024-04-25T11:06:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T11:06:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:Generating grub configuration file ... 2024-04-25T11:06:13.544 INFO:teuthology.orchestra.run.smithi106.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T11:06:13.564 INFO:teuthology.orchestra.run.smithi106.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T11:06:13.896 INFO:teuthology.orchestra.run.smithi106.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T11:06:13.902 INFO:teuthology.orchestra.run.smithi106.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T11:06:13.976 INFO:teuthology.orchestra.run.smithi106.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T11:06:13.981 INFO:teuthology.orchestra.run.smithi106.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T11:06:14.156 INFO:teuthology.orchestra.run.smithi106.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T11:06:14.156 INFO:teuthology.orchestra.run.smithi106.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T11:06:14.156 INFO:teuthology.orchestra.run.smithi106.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T11:06:14.203 INFO:teuthology.orchestra.run.smithi106.stdout:done 2024-04-25T11:06:14.411 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-04-25T11:06:14.431 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T11:06:14.491 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-04-25T11:06:14.691 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-04-25T11:06:14.691 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-04-25T11:06:14.694 DEBUG:teuthology.orchestra.run.smithi106:> dpkg -s linux-image-generic 2024-04-25T11:06:14.716 INFO:teuthology.orchestra.run.smithi106.stdout:Package: linux-image-generic 2024-04-25T11:06:14.716 INFO:teuthology.orchestra.run.smithi106.stdout:Status: install ok installed 2024-04-25T11:06:14.716 INFO:teuthology.orchestra.run.smithi106.stdout:Priority: optional 2024-04-25T11:06:14.716 INFO:teuthology.orchestra.run.smithi106.stdout:Section: kernel 2024-04-25T11:06:14.716 INFO:teuthology.orchestra.run.smithi106.stdout:Installed-Size: 21 2024-04-25T11:06:14.716 INFO:teuthology.orchestra.run.smithi106.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: amd64 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout:Source: linux-meta 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout:Version: 5.15.0.105.102 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout:Recommends: thermald 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout:Description: Generic Linux kernel image 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout: This package will always depend on the latest generic kernel image 2024-04-25T11:06:14.717 INFO:teuthology.orchestra.run.smithi106.stdout: available. 2024-04-25T11:06:14.718 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T11:06:14.718 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T11:06:14.718 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T11:06:14.718 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T11:06:14.718 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi106.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T11:06:14.718 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get clean 2024-04-25T11:06:14.864 INFO:teuthology.orchestra.run.smithi134.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T11:06:14.864 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T11:06:14.864 INFO:teuthology.orchestra.run.smithi134.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T11:06:14.864 INFO:teuthology.orchestra.run.smithi134.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T11:06:14.864 INFO:teuthology.orchestra.run.smithi134.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T11:06:14.864 INFO:teuthology.orchestra.run.smithi134.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T11:06:14.865 INFO:teuthology.orchestra.run.smithi134.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T11:06:14.865 INFO:teuthology.orchestra.run.smithi134.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T11:06:14.866 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get update 2024-04-25T11:06:14.904 INFO:teuthology.orchestra.run.smithi134.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T11:06:14.905 DEBUG:teuthology.orchestra.run.smithi134:> dpkg -s linux-image-generic 2024-04-25T11:06:14.921 INFO:teuthology.orchestra.run.smithi134.stdout:Package: linux-image-generic 2024-04-25T11:06:14.921 INFO:teuthology.orchestra.run.smithi134.stdout:Status: install ok installed 2024-04-25T11:06:14.921 INFO:teuthology.orchestra.run.smithi134.stdout:Priority: optional 2024-04-25T11:06:14.921 INFO:teuthology.orchestra.run.smithi134.stdout:Section: kernel 2024-04-25T11:06:14.921 INFO:teuthology.orchestra.run.smithi134.stdout:Installed-Size: 21 2024-04-25T11:06:14.921 INFO:teuthology.orchestra.run.smithi134.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: amd64 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout:Source: linux-meta 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout:Version: 5.15.0.105.102 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout:Recommends: thermald 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout:Description: Generic Linux kernel image 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout: This package will always depend on the latest generic kernel image 2024-04-25T11:06:14.922 INFO:teuthology.orchestra.run.smithi134.stdout: available. 2024-04-25T11:06:14.923 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-04-25T11:06:14.969 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.isn2TnOyIY 2024-04-25T11:06:14.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub/grub.cfg /tmp/tmp.isn2TnOyIY 2024-04-25T11:06:14.980 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T11:06:15.039 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.isn2TnOyIY 2024-04-25T11:06:15.096 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T11:06:15.157 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.isn2TnOyIY is 10KB 2024-04-25T11:06:15.169 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.isn2TnOyIY 2024-04-25T11:06:15.174 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T11:06:15.193 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T11:06:15.229 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:06:15.229 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T11:06:15.230 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T11:06:15.259 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T11:06:15.316 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T11:06:15.316 DEBUG:teuthology.orchestra.run.smithi134:> sudo update-grub 2024-04-25T11:06:16.367 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2024-04-25T11:06:16.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T11:06:16.443 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2024-04-25T11:06:16.652 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2024-04-25T11:06:16.653 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2024-04-25T11:06:16.725 INFO:teuthology.orchestra.run.smithi134.stderr:Sourcing file `/etc/default/grub' 2024-04-25T11:06:16.732 INFO:teuthology.orchestra.run.smithi134.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T11:06:16.736 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2024-04-25T11:06:16.829 INFO:teuthology.orchestra.run.smithi106.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T11:06:16.829 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T11:06:16.829 INFO:teuthology.orchestra.run.smithi106.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T11:06:16.829 INFO:teuthology.orchestra.run.smithi106.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T11:06:16.829 INFO:teuthology.orchestra.run.smithi106.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T11:06:16.830 INFO:teuthology.orchestra.run.smithi106.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T11:06:16.830 INFO:teuthology.orchestra.run.smithi106.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T11:06:16.830 INFO:teuthology.orchestra.run.smithi106.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T11:06:16.871 INFO:teuthology.orchestra.run.smithi106.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T11:06:16.874 DEBUG:teuthology.orchestra.run.smithi106:> dpkg -s linux-image-generic 2024-04-25T11:06:16.931 INFO:teuthology.orchestra.run.smithi106.stdout:Package: linux-image-generic 2024-04-25T11:06:16.931 INFO:teuthology.orchestra.run.smithi106.stdout:Status: install ok installed 2024-04-25T11:06:16.931 INFO:teuthology.orchestra.run.smithi106.stdout:Priority: optional 2024-04-25T11:06:16.931 INFO:teuthology.orchestra.run.smithi106.stdout:Section: kernel 2024-04-25T11:06:16.931 INFO:teuthology.orchestra.run.smithi106.stdout:Installed-Size: 21 2024-04-25T11:06:16.931 INFO:teuthology.orchestra.run.smithi106.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T11:06:16.931 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: amd64 2024-04-25T11:06:16.932 INFO:teuthology.orchestra.run.smithi106.stdout:Source: linux-meta 2024-04-25T11:06:16.932 INFO:teuthology.orchestra.run.smithi106.stdout:Version: 5.15.0.105.102 2024-04-25T11:06:16.932 INFO:teuthology.orchestra.run.smithi106.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T11:06:16.932 INFO:teuthology.orchestra.run.smithi106.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T11:06:16.932 INFO:teuthology.orchestra.run.smithi106.stdout:Recommends: thermald 2024-04-25T11:06:16.932 INFO:teuthology.orchestra.run.smithi106.stdout:Description: Generic Linux kernel image 2024-04-25T11:06:16.932 INFO:teuthology.orchestra.run.smithi106.stdout: This package will always depend on the latest generic kernel image 2024-04-25T11:06:16.932 INFO:teuthology.orchestra.run.smithi106.stdout: available. 2024-04-25T11:06:16.932 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-04-25T11:06:16.956 INFO:teuthology.orchestra.run.smithi134.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T11:06:16.961 INFO:teuthology.orchestra.run.smithi134.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T11:06:16.979 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.GUrk6phCyD 2024-04-25T11:06:16.980 DEBUG:teuthology.orchestra.run.smithi106:> sudo cp /boot/grub/grub.cfg /tmp/tmp.GUrk6phCyD 2024-04-25T11:06:17.036 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.GUrk6phCyD 2024-04-25T11:06:17.160 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.GUrk6phCyD is 10KB 2024-04-25T11:06:17.173 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.GUrk6phCyD 2024-04-25T11:06:17.178 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T11:06:17.231 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:06:17.231 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T11:06:17.231 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T11:06:17.261 INFO:teuthology.orchestra.run.smithi134.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T11:06:17.267 INFO:teuthology.orchestra.run.smithi134.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T11:06:17.294 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T11:06:17.294 DEBUG:teuthology.orchestra.run.smithi106:> sudo update-grub 2024-04-25T11:06:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T11:06:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T11:06:17.444 INFO:teuthology.orchestra.run.smithi134.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T11:06:17.444 INFO:teuthology.orchestra.run.smithi134.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T11:06:17.444 INFO:teuthology.orchestra.run.smithi134.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T11:06:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:done 2024-04-25T11:06:17.470 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2024-04-25T11:06:18.415 INFO:teuthology.orchestra.run.smithi106.stderr:Sourcing file `/etc/default/grub' 2024-04-25T11:06:18.416 INFO:teuthology.orchestra.run.smithi106.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T11:06:18.419 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2024-04-25T11:06:18.610 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T11:06:18.616 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T11:06:18.857 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T11:06:18.863 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T11:06:18.904 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T11:06:18.909 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T11:06:19.030 INFO:teuthology.orchestra.run.smithi106.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T11:06:19.030 INFO:teuthology.orchestra.run.smithi106.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T11:06:19.030 INFO:teuthology.orchestra.run.smithi106.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T11:06:19.047 INFO:teuthology.orchestra.run.smithi106.stderr:done 2024-04-25T11:06:19.057 DEBUG:teuthology.orchestra.run.smithi106:> sudo shutdown -r now 2024-04-25T11:06:47.542 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T11:06:47.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T11:06:47.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:06:49.060 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T11:06:49.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T11:06:49.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:07:05.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-04-25T11:07:07.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-04-25T11:07:14.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T11:07:14.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:07:16.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T11:07:16.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:07:17.954 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-04-25T11:07:19.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-04-25T11:07:29.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T11:07:29.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:07:31.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T11:07:31.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:07:48.387 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-04-25T11:07:49.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-04-25T11:08:03.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T11:08:03.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:08:03.751 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-04-25T11:08:04.873 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T11:08:04.873 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T11:08:04.873 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-04-25T11:08:04.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T11:08:04.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:08:04.923 INFO:teuthology.orchestra.run.smithi134.stdout:5.15.0-105-generic 2024-04-25T11:08:04.923 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T11:08:04.923 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T11:08:04.923 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T11:08:05.494 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-25T11:08:05.924 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T11:08:05.924 DEBUG:teuthology.orchestra.run.smithi134:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T11:08:06.173 INFO:teuthology.orchestra.run.smithi134.stdout:ttyS1 2024-04-25T11:08:06.213 DEBUG:teuthology.parallel:result is None 2024-04-25T11:08:06.300 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T11:08:06.300 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T11:08:06.300 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-25T11:08:06.347 INFO:teuthology.orchestra.run.smithi106.stdout:5.15.0-105-generic 2024-04-25T11:08:06.347 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T11:08:06.347 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T11:08:06.347 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T11:08:07.348 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T11:08:07.348 DEBUG:teuthology.orchestra.run.smithi106:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T11:08:07.490 INFO:teuthology.orchestra.run.smithi106.stdout:ttyS1 2024-04-25T11:08:07.512 DEBUG:teuthology.parallel:result is None 2024-04-25T11:08:07.513 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T11:08:07.520 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T11:08:07.521 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:08:07.522 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:08:07.528 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T11:08:07.535 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T11:08:07.543 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T11:08:07.543 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:08:07.572 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:08:07.590 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T11:08:07.596 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T11:08:07.597 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:08:07.624 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:08:07.638 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:07.645 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:07.657 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:07.667 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:07.669 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T11:08:07.675 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T11:08:07.676 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:08:07.688 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:08:07.722 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T11:08:07.731 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T11:08:07.731 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:08:07.744 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:08:07.770 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:08:07.795 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:08:07.843 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:08:07.843 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:08:07.911 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:08:07.917 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:08:07.967 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:08:07.967 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:08:08.049 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-04-25T11:08:08.051 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-04-25T11:08:08.125 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T11:08:08.133 INFO:teuthology.task.internal:Starting timer... 2024-04-25T11:08:08.133 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T11:08:08.144 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T11:08:08.153 DEBUG:teuthology.task.selinux:Excluding smithi106: OS 'ubuntu' does not support SELinux 2024-04-25T11:08:08.153 DEBUG:teuthology.task.selinux:Excluding smithi134: OS 'ubuntu' does not support SELinux 2024-04-25T11:08:08.153 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T11:08:08.154 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T11:08:08.154 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T11:08:08.154 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T11:08:08.162 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T11:08:08.168 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T11:08:08.331 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T11:08:08.346 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T11:08:08.348 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi106.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T11:15:14.769 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-04-25T11:15:14.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T11:15:14.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:15:14.857 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-25T11:15:14.934 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T11:15:14.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T11:15:14.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:15:15.019 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-04-25T11:15:15.102 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T11:15:15.103 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T11:15:15.116 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T11:15:15.116 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:15:15.117 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:15:15.119 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:15:15.119 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:15:15.146 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T11:15:15.146 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Command line: ntpd -gq 2024-04-25T11:15:15.146 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: ---------------------------------------------------- 2024-04-25T11:15:15.146 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T11:15:15.146 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T11:15:15.146 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: corporation. Support and training for ntp-4 are 2024-04-25T11:15:15.146 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: available at https://www.nwtime.org/support 2024-04-25T11:15:15.146 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: ---------------------------------------------------- 2024-04-25T11:15:15.147 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: proto: precision = 0.045 usec (-24) 2024-04-25T11:15:15.147 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: basedate set to 2022-02-04 2024-04-25T11:15:15.148 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: gps base set to 2022-02-06 (week 2196) 2024-04-25T11:15:15.148 INFO:teuthology.orchestra.run.smithi106.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T11:15:15.148 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T11:15:15.148 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T11:15:15.149 INFO:teuthology.orchestra.run.smithi106.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T11:15:15.150 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T11:15:15.150 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T11:15:15.150 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T11:15:15.150 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Listen normally on 3 enp3s0f1 172.21.15.106:123 2024-04-25T11:15:15.150 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Listen normally on 4 lo [::1]:123 2024-04-25T11:15:15.150 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce6d%5]:123 2024-04-25T11:15:15.150 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:15 ntpd[17199]: Listening on routing socket on fd #22 for interface updates 2024-04-25T11:15:15.173 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Command line: ntpd -gq 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: ---------------------------------------------------- 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: corporation. Support and training for ntp-4 are 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: available at https://www.nwtime.org/support 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: ---------------------------------------------------- 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: proto: precision = 0.044 usec (-24) 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: basedate set to 2022-02-04 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: gps base set to 2022-02-06 (week 2196) 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi134.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T11:15:15.176 INFO:teuthology.orchestra.run.smithi134.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T11:15:15.177 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T11:15:15.177 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T11:15:15.177 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T11:15:15.177 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Listen normally on 3 enp3s0f1 172.21.15.134:123 2024-04-25T11:15:15.177 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Listen normally on 4 lo [::1]:123 2024-04-25T11:15:15.177 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6de9%5]:123 2024-04-25T11:15:15.177 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:15 ntpd[17195]: Listening on routing socket on fd #22 for interface updates 2024-04-25T11:15:16.146 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:16 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:16.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:16 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:16.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:16 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:16.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:16 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:16.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:16 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:16.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:16 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:16.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:16 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:16.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:16 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:16.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:16 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:16.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:16 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:16.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:16 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:16.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:16 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:16.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:16 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:16.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:16 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:17.146 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:17 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:17.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:17 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:17.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:17 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:17.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:17 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:17.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:17 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:17.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:17 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:17.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:17 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:17.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:17 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:18 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:18.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:18 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:18.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:18 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:18.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:18 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:18.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:18 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:18.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:18 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:18.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:18.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:18.173 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:18 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:19.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:19 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:19.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:19 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:19.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:19 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:19.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:19 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:19.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:19 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:19.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:19 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:20.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:20 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:20.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:20.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:20.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:20.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:20.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:20.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:20 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:21.146 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:21 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:21.146 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:21 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:21 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:21 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:21 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:21 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:22.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.147 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:22.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:22.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:22.148 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:22.149 INFO:teuthology.orchestra.run.smithi106.stdout:25 Apr 11:15:22 ntpd[17199]: ntpd: time slew -0.002123 s 2024-04-25T11:15:22.149 INFO:teuthology.orchestra.run.smithi106.stdout:ntpd: time slew -0.002123s 2024-04-25T11:15:22.149 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T11:15:22.149 INFO:teuthology.orchestra.run.smithi106.stderr:25 Apr 11:15:22 ntpd[17199]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T11:15:22.174 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:22.175 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T11:15:22.176 INFO:teuthology.orchestra.run.smithi134.stdout:25 Apr 11:15:22 ntpd[17195]: ntpd: time slew -0.002478 s 2024-04-25T11:15:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:ntpd: time slew -0.002478s 2024-04-25T11:15:22.177 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T11:15:22.177 INFO:teuthology.orchestra.run.smithi134.stderr:25 Apr 11:15:22 ntpd[17195]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T11:15:22.197 INFO:teuthology.orchestra.run.smithi106.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T11:15:22.197 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================== 2024-04-25T11:15:22.197 INFO:teuthology.orchestra.run.smithi106.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:15:22.197 INFO:teuthology.orchestra.run.smithi106.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:15:22.197 INFO:teuthology.orchestra.run.smithi106.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:15:22.197 INFO:teuthology.orchestra.run.smithi106.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:15:22.314 INFO:teuthology.orchestra.run.smithi134.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T11:15:22.314 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================== 2024-04-25T11:15:22.314 INFO:teuthology.orchestra.run.smithi134.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:15:22.314 INFO:teuthology.orchestra.run.smithi134.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:15:22.315 INFO:teuthology.orchestra.run.smithi134.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:15:22.315 INFO:teuthology.orchestra.run.smithi134.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:15:22.316 INFO:teuthology.run_tasks:Running task install... 2024-04-25T11:15:22.339 DEBUG:teuthology.task.install:project ceph 2024-04-25T11:15:22.339 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T11:15:22.339 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T11:15:22.339 INFO:teuthology.task.install:Using flavor: default 2024-04-25T11:15:22.345 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T11:15:22.345 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T11:15:22.346 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-key list | grep Ceph 2024-04-25T11:15:22.347 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-key list | grep Ceph 2024-04-25T11:15:22.410 INFO:teuthology.orchestra.run.smithi106.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T11:15:22.423 INFO:teuthology.orchestra.run.smithi134.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T11:15:22.438 INFO:teuthology.orchestra.run.smithi106.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T11:15:22.438 INFO:teuthology.orchestra.run.smithi106.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T11:15:22.439 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T11:15:22.439 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T11:15:22.439 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:15:22.455 INFO:teuthology.orchestra.run.smithi134.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T11:15:22.455 INFO:teuthology.orchestra.run.smithi134.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T11:15:22.456 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T11:15:22.456 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T11:15:22.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:15:22.674 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default/ 2024-04-25T11:15:22.674 INFO:teuthology.task.install.deb:Package version is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:15:22.709 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default/ 2024-04-25T11:15:22.709 INFO:teuthology.task.install.deb:Package version is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:15:22.801 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:15:22.801 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T11:15:22.854 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get update 2024-04-25T11:15:22.900 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:15:22.900 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T11:15:22.914 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-04-25T11:15:23.082 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T11:15:23.082 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T11:15:23.082 INFO:teuthology.orchestra.run.smithi106.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T11:15:23.104 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T11:15:23.117 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T11:15:23.126 INFO:teuthology.orchestra.run.smithi106.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T11:15:23.140 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T11:15:23.170 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-25T11:15:23.177 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T11:15:23.190 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T11:15:23.201 INFO:teuthology.orchestra.run.smithi106.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-04-25T11:15:23.204 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T11:15:23.252 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T11:15:23.329 INFO:teuthology.orchestra.run.smithi134.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T11:15:23.396 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T11:15:23.433 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 29.2 kB in 0s (73.5 kB/s) 2024-04-25T11:15:23.768 INFO:teuthology.orchestra.run.smithi134.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T11:15:23.807 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-04-25T11:15:23.921 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-25T11:15:24.011 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 29.2 kB in 1s (32.3 kB/s) 2024-04-25T11:15:24.399 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2024-04-25T11:15:24.414 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3291-g994e2f20-1jammy cephadm=19.0.0-3291-g994e2f20-1jammy ceph-mds=19.0.0-3291-g994e2f20-1jammy ceph-mgr=19.0.0-3291-g994e2f20-1jammy ceph-common=19.0.0-3291-g994e2f20-1jammy ceph-fuse=19.0.0-3291-g994e2f20-1jammy ceph-test=19.0.0-3291-g994e2f20-1jammy ceph-volume=19.0.0-3291-g994e2f20-1jammy radosgw=19.0.0-3291-g994e2f20-1jammy python3-rados=19.0.0-3291-g994e2f20-1jammy python3-rgw=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy python3-rbd=19.0.0-3291-g994e2f20-1jammy libcephfs2=19.0.0-3291-g994e2f20-1jammy libcephfs-dev=19.0.0-3291-g994e2f20-1jammy librados2=19.0.0-3291-g994e2f20-1jammy librbd1=19.0.0-3291-g994e2f20-1jammy rbd-fuse=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy cephfs-shell=19.0.0-3291-g994e2f20-1jammy cephfs-top=19.0.0-3291-g994e2f20-1jammy cephfs-mirror=19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:15:24.474 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2024-04-25T11:15:24.669 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2024-04-25T11:15:24.669 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2024-04-25T11:15:24.868 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T11:15:24.868 INFO:teuthology.orchestra.run.smithi106.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T11:15:24.868 INFO:teuthology.orchestra.run.smithi106.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T11:15:24.868 INFO:teuthology.orchestra.run.smithi106.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T11:15:24.869 INFO:teuthology.orchestra.run.smithi106.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T11:15:24.869 INFO:teuthology.orchestra.run.smithi106.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T11:15:24.869 INFO:teuthology.orchestra.run.smithi106.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T11:15:24.869 INFO:teuthology.orchestra.run.smithi106.stdout:The following additional packages will be installed: 2024-04-25T11:15:24.870 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T11:15:24.870 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T11:15:24.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T11:15:24.872 INFO:teuthology.orchestra.run.smithi106.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout:Suggested packages: 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout:Recommended packages: 2024-04-25T11:15:24.873 INFO:teuthology.orchestra.run.smithi106.stdout: btrfs-tools 2024-04-25T11:15:24.979 INFO:teuthology.orchestra.run.smithi106.stdout:The following NEW packages will be installed: 2024-04-25T11:15:24.979 INFO:teuthology.orchestra.run.smithi106.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T11:15:24.979 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T11:15:24.979 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T11:15:24.979 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T11:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T11:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T11:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T11:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T11:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T11:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T11:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T11:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T11:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T11:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T11:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T11:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T11:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: socat unzip xmlstarlet zip 2024-04-25T11:15:25.030 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-04-25T11:15:25.047 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3291-g994e2f20-1jammy cephadm=19.0.0-3291-g994e2f20-1jammy ceph-mds=19.0.0-3291-g994e2f20-1jammy ceph-mgr=19.0.0-3291-g994e2f20-1jammy ceph-common=19.0.0-3291-g994e2f20-1jammy ceph-fuse=19.0.0-3291-g994e2f20-1jammy ceph-test=19.0.0-3291-g994e2f20-1jammy ceph-volume=19.0.0-3291-g994e2f20-1jammy radosgw=19.0.0-3291-g994e2f20-1jammy python3-rados=19.0.0-3291-g994e2f20-1jammy python3-rgw=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy python3-rbd=19.0.0-3291-g994e2f20-1jammy libcephfs2=19.0.0-3291-g994e2f20-1jammy libcephfs-dev=19.0.0-3291-g994e2f20-1jammy librados2=19.0.0-3291-g994e2f20-1jammy librbd1=19.0.0-3291-g994e2f20-1jammy rbd-fuse=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy cephfs-shell=19.0.0-3291-g994e2f20-1jammy cephfs-top=19.0.0-3291-g994e2f20-1jammy cephfs-mirror=19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:15:25.065 INFO:teuthology.orchestra.run.smithi106.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T11:15:25.065 INFO:teuthology.orchestra.run.smithi106.stdout:Need to get 175 MB of archives. 2024-04-25T11:15:25.065 INFO:teuthology.orchestra.run.smithi106.stdout:After this operation, 887 MB of additional disk space will be used. 2024-04-25T11:15:25.065 INFO:teuthology.orchestra.run.smithi106.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T11:15:25.103 INFO:teuthology.orchestra.run.smithi106.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T11:15:25.109 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-04-25T11:15:25.117 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T11:15:25.205 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T11:15:25.254 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T11:15:25.264 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T11:15:25.288 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T11:15:25.293 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3291-g994e2f20-1jammy [3,525 kB] 2024-04-25T11:15:25.311 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-04-25T11:15:25.312 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-04-25T11:15:25.395 INFO:teuthology.orchestra.run.smithi106.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T11:15:25.400 INFO:teuthology.orchestra.run.smithi106.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T11:15:25.419 INFO:teuthology.orchestra.run.smithi106.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T11:15:25.425 INFO:teuthology.orchestra.run.smithi106.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T11:15:25.427 INFO:teuthology.orchestra.run.smithi106.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T11:15:25.444 INFO:teuthology.orchestra.run.smithi106.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T11:15:25.447 INFO:teuthology.orchestra.run.smithi106.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T11:15:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T11:15:25.454 INFO:teuthology.orchestra.run.smithi106.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T11:15:25.456 INFO:teuthology.orchestra.run.smithi106.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T11:15:25.488 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T11:15:25.488 INFO:teuthology.orchestra.run.smithi134.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T11:15:25.488 INFO:teuthology.orchestra.run.smithi134.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T11:15:25.488 INFO:teuthology.orchestra.run.smithi134.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T11:15:25.488 INFO:teuthology.orchestra.run.smithi134.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T11:15:25.489 INFO:teuthology.orchestra.run.smithi134.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T11:15:25.489 INFO:teuthology.orchestra.run.smithi134.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T11:15:25.489 INFO:teuthology.orchestra.run.smithi134.stdout:The following additional packages will be installed: 2024-04-25T11:15:25.489 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T11:15:25.489 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T11:15:25.490 INFO:teuthology.orchestra.run.smithi134.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T11:15:25.490 INFO:teuthology.orchestra.run.smithi134.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T11:15:25.491 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T11:15:25.491 INFO:teuthology.orchestra.run.smithi134.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T11:15:25.491 INFO:teuthology.orchestra.run.smithi134.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T11:15:25.491 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T11:15:25.492 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout:Suggested packages: 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T11:15:25.493 INFO:teuthology.orchestra.run.smithi134.stdout:Recommended packages: 2024-04-25T11:15:25.494 INFO:teuthology.orchestra.run.smithi134.stdout: btrfs-tools 2024-04-25T11:15:25.504 INFO:teuthology.orchestra.run.smithi106.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T11:15:25.508 INFO:teuthology.orchestra.run.smithi106.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T11:15:25.511 INFO:teuthology.orchestra.run.smithi106.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T11:15:25.512 INFO:teuthology.orchestra.run.smithi106.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T11:15:25.512 INFO:teuthology.orchestra.run.smithi106.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T11:15:25.514 INFO:teuthology.orchestra.run.smithi106.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T11:15:25.515 INFO:teuthology.orchestra.run.smithi106.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T11:15:25.515 INFO:teuthology.orchestra.run.smithi106.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T11:15:25.516 INFO:teuthology.orchestra.run.smithi106.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T11:15:25.550 INFO:teuthology.orchestra.run.smithi106.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T11:15:25.555 INFO:teuthology.orchestra.run.smithi106.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T11:15:25.556 INFO:teuthology.orchestra.run.smithi106.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T11:15:25.561 INFO:teuthology.orchestra.run.smithi106.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T11:15:25.562 INFO:teuthology.orchestra.run.smithi106.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T11:15:25.563 INFO:teuthology.orchestra.run.smithi106.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T11:15:25.564 INFO:teuthology.orchestra.run.smithi106.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T11:15:25.566 INFO:teuthology.orchestra.run.smithi106.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T11:15:25.567 INFO:teuthology.orchestra.run.smithi106.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T11:15:25.583 INFO:teuthology.orchestra.run.smithi106.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T11:15:25.597 INFO:teuthology.orchestra.run.smithi134.stdout:The following NEW packages will be installed: 2024-04-25T11:15:25.597 INFO:teuthology.orchestra.run.smithi134.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T11:15:25.597 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T11:15:25.597 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T11:15:25.597 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T11:15:25.598 INFO:teuthology.orchestra.run.smithi106.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T11:15:25.599 INFO:teuthology.orchestra.run.smithi134.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T11:15:25.599 INFO:teuthology.orchestra.run.smithi134.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T11:15:25.599 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T11:15:25.599 INFO:teuthology.orchestra.run.smithi134.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T11:15:25.599 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T11:15:25.599 INFO:teuthology.orchestra.run.smithi134.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T11:15:25.599 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T11:15:25.600 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T11:15:25.601 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T11:15:25.601 INFO:teuthology.orchestra.run.smithi134.stdout: socat unzip xmlstarlet zip 2024-04-25T11:15:25.601 INFO:teuthology.orchestra.run.smithi106.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T11:15:25.601 INFO:teuthology.orchestra.run.smithi106.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T11:15:25.611 INFO:teuthology.orchestra.run.smithi106.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T11:15:25.611 INFO:teuthology.orchestra.run.smithi106.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T11:15:25.612 INFO:teuthology.orchestra.run.smithi106.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T11:15:25.613 INFO:teuthology.orchestra.run.smithi106.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T11:15:25.616 INFO:teuthology.orchestra.run.smithi106.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T11:15:25.620 INFO:teuthology.orchestra.run.smithi106.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T11:15:25.635 INFO:teuthology.orchestra.run.smithi106.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T11:15:25.638 INFO:teuthology.orchestra.run.smithi106.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T11:15:25.640 INFO:teuthology.orchestra.run.smithi106.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T11:15:25.644 INFO:teuthology.orchestra.run.smithi106.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T11:15:25.645 INFO:teuthology.orchestra.run.smithi106.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T11:15:25.658 INFO:teuthology.orchestra.run.smithi106.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T11:15:25.662 INFO:teuthology.orchestra.run.smithi106.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T11:15:25.688 INFO:teuthology.orchestra.run.smithi134.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T11:15:25.688 INFO:teuthology.orchestra.run.smithi134.stdout:Need to get 175 MB of archives. 2024-04-25T11:15:25.688 INFO:teuthology.orchestra.run.smithi134.stdout:After this operation, 887 MB of additional disk space will be used. 2024-04-25T11:15:25.688 INFO:teuthology.orchestra.run.smithi134.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T11:15:25.742 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3291-g994e2f20-1jammy [3,525 kB] 2024-04-25T11:15:25.744 INFO:teuthology.orchestra.run.smithi106.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T11:15:25.746 INFO:teuthology.orchestra.run.smithi106.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T11:15:25.751 INFO:teuthology.orchestra.run.smithi106.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T11:15:25.751 INFO:teuthology.orchestra.run.smithi106.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T11:15:25.751 INFO:teuthology.orchestra.run.smithi106.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T11:15:25.752 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T11:15:25.770 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T11:15:25.783 INFO:teuthology.orchestra.run.smithi106.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T11:15:25.786 INFO:teuthology.orchestra.run.smithi106.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T11:15:25.786 INFO:teuthology.orchestra.run.smithi106.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T11:15:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T11:15:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T11:15:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T11:15:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T11:15:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T11:15:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T11:15:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T11:15:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T11:15:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T11:15:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T11:15:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T11:15:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T11:15:25.905 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T11:15:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T11:15:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T11:15:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T11:15:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T11:15:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T11:15:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T11:15:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T11:15:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T11:15:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T11:15:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T11:15:25.985 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T11:15:25.992 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T11:15:25.999 INFO:teuthology.orchestra.run.smithi106.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T11:15:26.001 INFO:teuthology.orchestra.run.smithi106.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T11:15:26.002 INFO:teuthology.orchestra.run.smithi106.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T11:15:26.012 INFO:teuthology.orchestra.run.smithi106.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T11:15:26.013 INFO:teuthology.orchestra.run.smithi106.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T11:15:26.013 INFO:teuthology.orchestra.run.smithi106.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T11:15:26.016 INFO:teuthology.orchestra.run.smithi106.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T11:15:26.041 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T11:15:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T11:15:26.214 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T11:15:26.224 INFO:teuthology.orchestra.run.smithi134.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T11:15:26.248 INFO:teuthology.orchestra.run.smithi134.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T11:15:26.256 INFO:teuthology.orchestra.run.smithi134.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T11:15:26.257 INFO:teuthology.orchestra.run.smithi134.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T11:15:26.286 INFO:teuthology.orchestra.run.smithi134.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T11:15:26.290 INFO:teuthology.orchestra.run.smithi134.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T11:15:26.295 INFO:teuthology.orchestra.run.smithi134.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T11:15:26.297 INFO:teuthology.orchestra.run.smithi134.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T11:15:26.298 INFO:teuthology.orchestra.run.smithi134.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T11:15:26.324 INFO:teuthology.orchestra.run.smithi134.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T11:15:26.329 INFO:teuthology.orchestra.run.smithi134.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T11:15:26.333 INFO:teuthology.orchestra.run.smithi134.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T11:15:26.334 INFO:teuthology.orchestra.run.smithi134.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T11:15:26.365 INFO:teuthology.orchestra.run.smithi134.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T11:15:26.367 INFO:teuthology.orchestra.run.smithi134.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T11:15:26.368 INFO:teuthology.orchestra.run.smithi134.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T11:15:26.369 INFO:teuthology.orchestra.run.smithi134.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T11:15:26.370 INFO:teuthology.orchestra.run.smithi134.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T11:15:26.371 INFO:teuthology.orchestra.run.smithi134.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T11:15:26.402 INFO:teuthology.orchestra.run.smithi134.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T11:15:26.403 INFO:teuthology.orchestra.run.smithi134.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T11:15:26.409 INFO:teuthology.orchestra.run.smithi134.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T11:15:26.410 INFO:teuthology.orchestra.run.smithi134.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T11:15:26.439 INFO:teuthology.orchestra.run.smithi134.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T11:15:26.440 INFO:teuthology.orchestra.run.smithi134.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T11:15:26.442 INFO:teuthology.orchestra.run.smithi134.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T11:15:26.443 INFO:teuthology.orchestra.run.smithi134.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T11:15:26.444 INFO:teuthology.orchestra.run.smithi134.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T11:15:26.446 INFO:teuthology.orchestra.run.smithi134.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T11:15:26.473 INFO:teuthology.orchestra.run.smithi134.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T11:15:26.474 INFO:teuthology.orchestra.run.smithi134.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T11:15:26.485 INFO:teuthology.orchestra.run.smithi134.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T11:15:26.485 INFO:teuthology.orchestra.run.smithi134.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T11:15:26.510 INFO:teuthology.orchestra.run.smithi134.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T11:15:26.511 INFO:teuthology.orchestra.run.smithi134.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T11:15:26.513 INFO:teuthology.orchestra.run.smithi134.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T11:15:26.517 INFO:teuthology.orchestra.run.smithi134.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T11:15:26.519 INFO:teuthology.orchestra.run.smithi134.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T11:15:26.522 INFO:teuthology.orchestra.run.smithi134.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T11:15:26.548 INFO:teuthology.orchestra.run.smithi134.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T11:15:26.553 INFO:teuthology.orchestra.run.smithi134.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T11:15:26.554 INFO:teuthology.orchestra.run.smithi134.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T11:15:26.556 INFO:teuthology.orchestra.run.smithi134.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T11:15:26.597 INFO:teuthology.orchestra.run.smithi134.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T11:15:26.781 INFO:teuthology.orchestra.run.smithi134.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T11:15:26.782 INFO:teuthology.orchestra.run.smithi134.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T11:15:26.784 INFO:teuthology.orchestra.run.smithi134.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T11:15:26.785 INFO:teuthology.orchestra.run.smithi134.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T11:15:26.786 INFO:teuthology.orchestra.run.smithi134.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T11:15:26.810 INFO:teuthology.orchestra.run.smithi134.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T11:15:26.812 INFO:teuthology.orchestra.run.smithi134.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T11:15:26.812 INFO:teuthology.orchestra.run.smithi134.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T11:15:26.830 INFO:teuthology.orchestra.run.smithi134.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3291-g994e2f20-1jammy [3,287 kB] 2024-04-25T11:15:26.837 INFO:teuthology.orchestra.run.smithi134.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T11:15:26.838 INFO:teuthology.orchestra.run.smithi134.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T11:15:26.849 INFO:teuthology.orchestra.run.smithi134.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T11:15:26.850 INFO:teuthology.orchestra.run.smithi134.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T11:15:26.850 INFO:teuthology.orchestra.run.smithi134.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T11:15:26.851 INFO:teuthology.orchestra.run.smithi134.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T11:15:26.851 INFO:teuthology.orchestra.run.smithi134.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T11:15:26.852 INFO:teuthology.orchestra.run.smithi134.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T11:15:26.855 INFO:teuthology.orchestra.run.smithi134.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T11:15:26.870 INFO:teuthology.orchestra.run.smithi134.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T11:15:26.871 INFO:teuthology.orchestra.run.smithi134.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T11:15:26.872 INFO:teuthology.orchestra.run.smithi134.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T11:15:26.895 INFO:teuthology.orchestra.run.smithi134.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T11:15:26.899 INFO:teuthology.orchestra.run.smithi134.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T11:15:26.899 INFO:teuthology.orchestra.run.smithi134.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T11:15:26.901 INFO:teuthology.orchestra.run.smithi134.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T11:15:26.901 INFO:teuthology.orchestra.run.smithi134.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T11:15:26.908 INFO:teuthology.orchestra.run.smithi134.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T11:15:26.910 INFO:teuthology.orchestra.run.smithi134.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T11:15:26.930 INFO:teuthology.orchestra.run.smithi134.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T11:15:26.930 INFO:teuthology.orchestra.run.smithi134.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T11:15:26.931 INFO:teuthology.orchestra.run.smithi134.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T11:15:26.963 INFO:teuthology.orchestra.run.smithi134.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T11:15:26.966 INFO:teuthology.orchestra.run.smithi134.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T11:15:26.967 INFO:teuthology.orchestra.run.smithi134.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T11:15:26.979 INFO:teuthology.orchestra.run.smithi134.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T11:15:26.979 INFO:teuthology.orchestra.run.smithi134.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T11:15:26.980 INFO:teuthology.orchestra.run.smithi134.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T11:15:26.983 INFO:teuthology.orchestra.run.smithi134.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T11:15:26.995 INFO:teuthology.orchestra.run.smithi134.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T11:15:27.147 INFO:teuthology.orchestra.run.smithi134.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3291-g994e2f20-1jammy [726 kB] 2024-04-25T11:15:27.189 INFO:teuthology.orchestra.run.smithi134.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3291-g994e2f20-1jammy [356 kB] 2024-04-25T11:15:27.209 INFO:teuthology.orchestra.run.smithi134.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3291-g994e2f20-1jammy [32.8 kB] 2024-04-25T11:15:27.210 INFO:teuthology.orchestra.run.smithi134.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3291-g994e2f20-1jammy [180 kB] 2024-04-25T11:15:27.226 INFO:teuthology.orchestra.run.smithi134.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3291-g994e2f20-1jammy [67.1 kB] 2024-04-25T11:15:27.237 INFO:teuthology.orchestra.run.smithi134.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3291-g994e2f20-1jammy [330 kB] 2024-04-25T11:15:27.278 INFO:teuthology.orchestra.run.smithi134.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3291-g994e2f20-1jammy [7,189 kB] 2024-04-25T11:15:27.628 INFO:teuthology.orchestra.run.smithi134.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3291-g994e2f20-1jammy [112 kB] 2024-04-25T11:15:27.630 INFO:teuthology.orchestra.run.smithi134.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3291-g994e2f20-1jammy [471 kB] 2024-04-25T11:15:27.647 INFO:teuthology.orchestra.run.smithi134.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3291-g994e2f20-1jammy [25.6 MB] 2024-04-25T11:15:28.102 INFO:teuthology.orchestra.run.smithi106.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3291-g994e2f20-1jammy [3,287 kB] 2024-04-25T11:15:28.679 INFO:teuthology.orchestra.run.smithi134.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3291-g994e2f20-1jammy [4,917 kB] 2024-04-25T11:15:28.754 INFO:teuthology.orchestra.run.smithi134.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3291-g994e2f20-1jammy [244 kB] 2024-04-25T11:15:28.759 INFO:teuthology.orchestra.run.smithi134.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3291-g994e2f20-1jammy [125 kB] 2024-04-25T11:15:28.763 INFO:teuthology.orchestra.run.smithi134.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3291-g994e2f20-1jammy [1,542 kB] 2024-04-25T11:15:28.901 INFO:teuthology.orchestra.run.smithi134.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3291-g994e2f20-1jammy [6,148 kB] 2024-04-25T11:15:29.045 INFO:teuthology.orchestra.run.smithi134.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3291-g994e2f20-1jammy [22.5 MB] 2024-04-25T11:15:29.151 INFO:teuthology.orchestra.run.smithi106.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3291-g994e2f20-1jammy [726 kB] 2024-04-25T11:15:29.406 INFO:teuthology.orchestra.run.smithi106.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3291-g994e2f20-1jammy [356 kB] 2024-04-25T11:15:29.554 INFO:teuthology.orchestra.run.smithi106.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3291-g994e2f20-1jammy [32.8 kB] 2024-04-25T11:15:29.569 INFO:teuthology.orchestra.run.smithi106.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3291-g994e2f20-1jammy [180 kB] 2024-04-25T11:15:29.609 INFO:teuthology.orchestra.run.smithi106.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3291-g994e2f20-1jammy [67.1 kB] 2024-04-25T11:15:29.657 INFO:teuthology.orchestra.run.smithi106.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3291-g994e2f20-1jammy [330 kB] 2024-04-25T11:15:29.744 INFO:teuthology.orchestra.run.smithi106.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3291-g994e2f20-1jammy [7,189 kB] 2024-04-25T11:15:29.973 INFO:teuthology.orchestra.run.smithi134.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3291-g994e2f20-1jammy [14.1 kB] 2024-04-25T11:15:29.974 INFO:teuthology.orchestra.run.smithi134.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3291-g994e2f20-1jammy [856 kB] 2024-04-25T11:15:29.990 INFO:teuthology.orchestra.run.smithi134.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3291-g994e2f20-1jammy [2,367 kB] 2024-04-25T11:15:30.031 INFO:teuthology.orchestra.run.smithi134.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3291-g994e2f20-1jammy [791 kB] 2024-04-25T11:15:30.041 INFO:teuthology.orchestra.run.smithi134.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3291-g994e2f20-1jammy [148 kB] 2024-04-25T11:15:30.043 INFO:teuthology.orchestra.run.smithi134.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3291-g994e2f20-1jammy [2,297 kB] 2024-04-25T11:15:30.112 INFO:teuthology.orchestra.run.smithi134.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3291-g994e2f20-1jammy [8,625 kB] 2024-04-25T11:15:30.440 INFO:teuthology.orchestra.run.smithi134.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3291-g994e2f20-1jammy [14.2 kB] 2024-04-25T11:15:30.440 INFO:teuthology.orchestra.run.smithi134.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3291-g994e2f20-1jammy [44.9 MB] 2024-04-25T11:15:31.617 INFO:teuthology.orchestra.run.smithi106.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3291-g994e2f20-1jammy [112 kB] 2024-04-25T11:15:31.648 INFO:teuthology.orchestra.run.smithi106.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3291-g994e2f20-1jammy [471 kB] 2024-04-25T11:15:31.766 INFO:teuthology.orchestra.run.smithi106.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3291-g994e2f20-1jammy [25.6 MB] 2024-04-25T11:15:32.376 INFO:teuthology.orchestra.run.smithi134.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3291-g994e2f20-1jammy [124 kB] 2024-04-25T11:15:32.378 INFO:teuthology.orchestra.run.smithi134.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3291-g994e2f20-1jammy [236 kB] 2024-04-25T11:15:32.383 INFO:teuthology.orchestra.run.smithi134.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3291-g994e2f20-1jammy [33.9 kB] 2024-04-25T11:15:32.384 INFO:teuthology.orchestra.run.smithi134.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3291-g994e2f20-1jammy [23.4 kB] 2024-04-25T11:15:32.436 INFO:teuthology.orchestra.run.smithi134.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3291-g994e2f20-1jammy [39.0 kB] 2024-04-25T11:15:32.437 INFO:teuthology.orchestra.run.smithi134.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3291-g994e2f20-1jammy [14.2 MB] 2024-04-25T11:15:32.866 INFO:teuthology.orchestra.run.smithi134.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3291-g994e2f20-1jammy [92.6 kB] 2024-04-25T11:15:33.488 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 175 MB in 7s (24.1 MB/s) 2024-04-25T11:15:33.628 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T11:15:33.727 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T11:15:33.733 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T11:15:33.776 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T11:15:34.078 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T11:15:34.105 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T11:15:34.146 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T11:15:34.414 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T11:15:34.430 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T11:15:34.464 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T11:15:34.775 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T11:15:34.802 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T11:15:34.893 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T11:15:35.254 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T11:15:35.281 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T11:15:35.321 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T11:15:35.657 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T11:15:35.684 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T11:15:35.724 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T11:15:35.905 INFO:teuthology.orchestra.run.smithi106.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3291-g994e2f20-1jammy [4,917 kB] 2024-04-25T11:15:36.111 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T11:15:36.138 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T11:15:36.186 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:15:36.564 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T11:15:36.590 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T11:15:36.630 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:15:36.827 INFO:teuthology.orchestra.run.smithi106.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3291-g994e2f20-1jammy [244 kB] 2024-04-25T11:15:36.834 INFO:teuthology.orchestra.run.smithi106.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3291-g994e2f20-1jammy [125 kB] 2024-04-25T11:15:36.879 INFO:teuthology.orchestra.run.smithi106.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3291-g994e2f20-1jammy [1,542 kB] 2024-04-25T11:15:36.983 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T11:15:37.010 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T11:15:37.049 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:15:37.121 INFO:teuthology.orchestra.run.smithi106.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3291-g994e2f20-1jammy [6,148 kB] 2024-04-25T11:15:37.361 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T11:15:37.384 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T11:15:37.419 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T11:15:37.747 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librados2. 2024-04-25T11:15:37.774 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../010-librados2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:37.814 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:38.275 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librbd1. 2024-04-25T11:15:38.301 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../011-librbd1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:38.341 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi106.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3291-g994e2f20-1jammy [22.5 MB] 2024-04-25T11:15:38.794 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T11:15:38.821 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:38.852 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:39.097 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rados. 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:39.155 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:39.507 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T11:15:39.534 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:15:39.574 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:39.826 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T11:15:39.842 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:39.885 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:40.213 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T11:15:40.239 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:15:40.279 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:40.590 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T11:15:40.617 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T11:15:40.648 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T11:15:40.892 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T11:15:40.908 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:40.959 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:41.321 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T11:15:41.347 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T11:15:41.387 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T11:15:41.673 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T11:15:41.700 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T11:15:41.740 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T11:15:42.026 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T11:15:42.041 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T11:15:42.076 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T11:15:42.358 INFO:teuthology.orchestra.run.smithi106.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3291-g994e2f20-1jammy [14.1 kB] 2024-04-25T11:15:42.358 INFO:teuthology.orchestra.run.smithi106.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3291-g994e2f20-1jammy [856 kB] 2024-04-25T11:15:42.414 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package lua5.1. 2024-04-25T11:15:42.441 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T11:15:42.481 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T11:15:42.538 INFO:teuthology.orchestra.run.smithi106.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3291-g994e2f20-1jammy [2,367 kB] 2024-04-25T11:15:42.817 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package lua-any. 2024-04-25T11:15:42.844 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T11:15:42.883 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T11:15:43.136 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package zip. 2024-04-25T11:15:43.163 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T11:15:43.202 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T11:15:43.237 INFO:teuthology.orchestra.run.smithi106.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3291-g994e2f20-1jammy [791 kB] 2024-04-25T11:15:43.428 INFO:teuthology.orchestra.run.smithi106.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3291-g994e2f20-1jammy [148 kB] 2024-04-25T11:15:43.470 INFO:teuthology.orchestra.run.smithi106.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3291-g994e2f20-1jammy [2,297 kB] 2024-04-25T11:15:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package unzip. 2024-04-25T11:15:43.516 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T11:15:43.547 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T11:15:44.048 INFO:teuthology.orchestra.run.smithi106.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3291-g994e2f20-1jammy [8,625 kB] 2024-04-25T11:15:44.335 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package luarocks. 2024-04-25T11:15:44.362 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T11:15:44.401 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T11:15:44.776 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librgw2. 2024-04-25T11:15:44.803 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../027-librgw2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:44.842 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:45.444 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T11:15:45.471 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:45.511 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:45.856 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T11:15:45.882 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T11:15:45.964 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T11:15:46.267 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T11:15:46.293 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:46.333 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:46.416 INFO:teuthology.orchestra.run.smithi106.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3291-g994e2f20-1jammy [14.2 kB] 2024-04-25T11:15:46.430 INFO:teuthology.orchestra.run.smithi106.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3291-g994e2f20-1jammy [44.9 MB] 2024-04-25T11:15:46.695 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-common. 2024-04-25T11:15:46.710 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:46.744 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:48.258 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-base. 2024-04-25T11:15:48.284 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:15:48.334 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:15:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T11:15:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T11:15:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T11:15:49.565 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T11:15:49.592 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T11:15:49.631 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T11:15:49.926 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T11:15:49.942 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T11:15:49.976 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T11:15:50.271 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T11:15:50.299 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T11:15:50.354 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T11:15:50.657 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T11:15:50.683 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T11:15:50.723 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T11:15:51.001 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T11:15:51.028 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T11:15:51.067 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T11:15:51.312 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-portend. 2024-04-25T11:15:51.339 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T11:15:51.378 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T11:15:51.681 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T11:15:51.708 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T11:15:51.747 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T11:15:52.051 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T11:15:52.077 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T11:15:52.117 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T11:15:52.479 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T11:15:52.505 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T11:15:52.546 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T11:15:52.873 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T11:15:52.900 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T11:15:52.940 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T11:15:53.251 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T11:15:53.278 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T11:15:53.318 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T11:15:53.629 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-mako. 2024-04-25T11:15:53.655 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T11:15:53.695 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T11:15:53.999 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T11:15:54.025 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T11:15:54.065 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T11:15:54.334 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T11:15:54.361 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T11:15:54.401 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T11:15:54.703 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-webob. 2024-04-25T11:15:54.730 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T11:15:54.779 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T11:15:55.123 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T11:15:55.149 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T11:15:55.193 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T11:15:55.509 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T11:15:55.535 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T11:15:55.576 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T11:15:55.870 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-paste. 2024-04-25T11:15:55.897 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T11:15:55.936 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T11:15:56.290 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T11:15:56.317 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T11:15:56.356 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T11:15:56.659 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T11:15:56.685 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T11:15:56.726 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T11:15:57.004 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T11:15:57.030 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T11:15:57.070 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T11:15:57.238 INFO:teuthology.orchestra.run.smithi106.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3291-g994e2f20-1jammy [124 kB] 2024-04-25T11:15:57.240 INFO:teuthology.orchestra.run.smithi106.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3291-g994e2f20-1jammy [236 kB] 2024-04-25T11:15:57.246 INFO:teuthology.orchestra.run.smithi106.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3291-g994e2f20-1jammy [33.9 kB] 2024-04-25T11:15:57.246 INFO:teuthology.orchestra.run.smithi106.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3291-g994e2f20-1jammy [23.4 kB] 2024-04-25T11:15:57.247 INFO:teuthology.orchestra.run.smithi106.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3291-g994e2f20-1jammy [39.0 kB] 2024-04-25T11:15:57.248 INFO:teuthology.orchestra.run.smithi106.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3291-g994e2f20-1jammy [14.2 MB] 2024-04-25T11:15:57.332 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T11:15:57.358 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T11:15:57.390 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T11:15:57.686 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T11:15:57.713 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T11:15:57.752 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T11:15:58.097 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-idna. 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T11:15:58.164 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T11:15:59.333 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T11:15:59.360 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T11:15:59.400 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T11:15:59.736 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-requests. 2024-04-25T11:15:59.763 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T11:15:59.803 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T11:16:00.089 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T11:16:00.104 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T11:16:00.139 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T11:16:00.321 INFO:teuthology.orchestra.run.smithi106.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3291-g994e2f20-1jammy [92.6 kB] 2024-04-25T11:16:00.475 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T11:16:00.502 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:00.550 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:00.946 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 175 MB in 35s (4,946 kB/s) 2024-04-25T11:16:00.955 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T11:16:00.982 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:01.021 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:01.064 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T11:16:01.152 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T11:16:01.158 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T11:16:01.205 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T11:16:01.316 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T11:16:01.342 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T11:16:01.382 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T11:16:01.449 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T11:16:01.475 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T11:16:01.508 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T11:16:01.652 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T11:16:01.678 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T11:16:01.710 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T11:16:01.819 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T11:16:01.845 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T11:16:01.885 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T11:16:02.030 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T11:16:02.056 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:02.096 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:02.230 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T11:16:02.257 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T11:16:02.339 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T11:16:02.499 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T11:16:02.526 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:02.591 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:02.692 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T11:16:02.718 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T11:16:02.759 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T11:16:03.045 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T11:16:03.072 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T11:16:03.103 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T11:16:03.178 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T11:16:03.204 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:03.244 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:03.566 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T11:16:03.592 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T11:16:03.641 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:16:04.052 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T11:16:04.078 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T11:16:04.118 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:16:04.330 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph. 2024-04-25T11:16:04.346 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../068-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:04.389 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:04.413 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T11:16:04.439 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T11:16:04.471 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:16:04.691 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T11:16:04.717 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:04.758 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:04.816 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T11:16:04.843 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T11:16:04.883 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T11:16:05.119 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T11:16:05.145 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:05.185 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:05.202 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librados2. 2024-04-25T11:16:05.229 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../010-librados2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:05.269 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:05.564 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package cephadm. 2024-04-25T11:16:05.590 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../071-cephadm_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:05.621 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:05.897 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librbd1. 2024-04-25T11:16:05.923 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../011-librbd1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:05.932 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T11:16:05.955 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:05.959 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T11:16:05.999 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T11:16:06.302 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T11:16:06.317 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T11:16:06.351 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T11:16:06.425 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T11:16:06.452 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:06.491 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:06.769 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rados. 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:07.163 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T11:16:07.190 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:07.230 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:07.340 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-babel. 2024-04-25T11:16:07.367 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T11:16:07.398 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T11:16:07.458 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T11:16:07.473 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:07.507 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:07.751 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T11:16:07.778 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T11:16:07.810 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T11:16:07.817 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T11:16:07.836 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:07.878 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:08.170 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T11:16:08.198 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:08.205 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T11:16:08.228 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T11:16:08.236 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:08.271 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T11:16:08.508 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T11:16:08.534 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:08.574 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T11:16:08.574 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:08.602 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T11:16:08.640 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T11:16:08.852 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T11:16:08.879 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T11:16:08.910 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T11:16:08.934 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-routes. 2024-04-25T11:16:08.962 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T11:16:09.001 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T11:16:09.163 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T11:16:09.178 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T11:16:09.221 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T11:16:09.354 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T11:16:09.381 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:09.420 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:09.507 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T11:16:09.533 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T11:16:09.574 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T11:16:09.894 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package lua5.1. 2024-04-25T11:16:09.922 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T11:16:09.960 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T11:16:10.230 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package lua-any. 2024-04-25T11:16:10.245 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T11:16:10.280 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T11:16:10.550 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package zip. 2024-04-25T11:16:10.564 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T11:16:10.608 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T11:16:10.875 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T11:16:10.891 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T11:16:10.911 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package unzip. 2024-04-25T11:16:10.933 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T11:16:10.937 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T11:16:10.978 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T11:16:11.331 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package luarocks. 2024-04-25T11:16:11.358 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T11:16:11.390 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T11:16:11.454 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T11:16:11.481 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T11:16:11.520 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T11:16:11.739 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librgw2. 2024-04-25T11:16:11.765 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../027-librgw2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:11.806 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:11.832 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T11:16:11.859 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T11:16:11.975 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T11:16:12.287 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T11:16:12.314 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T11:16:12.353 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T11:16:12.458 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T11:16:12.473 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:12.517 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:12.853 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T11:16:12.880 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T11:16:12.919 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T11:16:13.020 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T11:16:13.048 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:13.087 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:13.189 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T11:16:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:13.668 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-common. 2024-04-25T11:16:13.694 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:13.734 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:13.998 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T11:16:14.026 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T11:16:14.056 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T11:16:14.343 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T11:16:14.370 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T11:16:14.409 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T11:16:14.746 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T11:16:14.774 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T11:16:14.812 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T11:16:15.124 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T11:16:15.151 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T11:16:15.189 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-base. 2024-04-25T11:16:15.190 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T11:16:15.216 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:15.257 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:15.502 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T11:16:15.530 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T11:16:15.568 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T11:16:15.880 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T11:16:15.885 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T11:16:15.908 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T11:16:15.912 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T11:16:15.938 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T11:16:15.952 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T11:16:16.242 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T11:16:16.269 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T11:16:16.308 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T11:16:16.430 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T11:16:16.456 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T11:16:16.496 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T11:16:16.636 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T11:16:16.663 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T11:16:16.702 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T11:16:16.808 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T11:16:16.835 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T11:16:16.874 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T11:16:17.102 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T11:16:17.117 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T11:16:17.152 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T11:16:17.358 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T11:16:17.386 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:17.424 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:17.455 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T11:16:17.481 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T11:16:17.521 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T11:16:17.735 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T11:16:17.763 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T11:16:17.802 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T11:16:17.826 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T11:16:17.853 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T11:16:17.893 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T11:16:18.106 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T11:16:18.132 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T11:16:18.154 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-portend. 2024-04-25T11:16:18.163 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T11:16:18.180 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T11:16:18.221 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T11:16:18.416 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package jq. 2024-04-25T11:16:18.435 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T11:16:18.474 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T11:16:18.523 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T11:16:18.550 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T11:16:18.590 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T11:16:18.727 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package socat. 2024-04-25T11:16:18.754 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T11:16:18.793 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T11:16:18.893 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T11:16:18.919 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T11:16:18.960 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T11:16:19.113 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T11:16:19.140 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T11:16:19.200 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T11:16:19.305 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T11:16:19.334 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T11:16:19.371 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T11:16:19.550 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-test. 2024-04-25T11:16:19.566 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:19.608 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:19.816 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T11:16:19.843 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T11:16:19.883 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T11:16:20.161 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T11:16:20.188 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T11:16:20.227 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T11:16:20.489 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-mako. 2024-04-25T11:16:20.515 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T11:16:20.558 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T11:16:20.867 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T11:16:20.894 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T11:16:20.925 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T11:16:21.236 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T11:16:21.252 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T11:16:21.311 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T11:16:21.606 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-webob. 2024-04-25T11:16:21.632 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T11:16:21.672 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T11:16:21.904 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T11:16:21.932 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:21.970 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:21.994 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T11:16:22.018 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T11:16:22.061 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T11:16:22.328 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T11:16:22.344 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T11:16:22.360 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T11:16:22.378 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T11:16:22.387 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:22.426 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:22.681 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-paste. 2024-04-25T11:16:22.704 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T11:16:22.708 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T11:16:22.720 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T11:16:22.747 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T11:16:22.754 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T11:16:22.998 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T11:16:23.014 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:23.034 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T11:16:23.050 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T11:16:23.057 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:23.084 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T11:16:23.318 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T11:16:23.345 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:23.379 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T11:16:23.384 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T11:16:23.445 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T11:16:23.637 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T11:16:23.653 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:23.695 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:23.740 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T11:16:23.766 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T11:16:23.798 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T11:16:23.990 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T11:16:24.017 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T11:16:24.047 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T11:16:24.084 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T11:16:24.111 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T11:16:24.150 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T11:16:24.336 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T11:16:24.363 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T11:16:24.402 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T11:16:24.489 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T11:16:24.516 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T11:16:24.555 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T11:16:24.748 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T11:16:24.775 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T11:16:24.814 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T11:16:24.900 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-idna. 2024-04-25T11:16:24.926 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T11:16:24.966 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T11:16:25.236 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package pkg-config. 2024-04-25T11:16:25.264 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T11:16:25.278 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T11:16:25.293 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T11:16:25.302 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T11:16:25.336 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T11:16:25.555 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T11:16:25.583 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T11:16:25.613 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T11:16:25.648 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-requests. 2024-04-25T11:16:25.674 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T11:16:25.714 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T11:16:26.026 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T11:16:26.052 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T11:16:26.075 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T11:16:26.092 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T11:16:26.103 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T11:16:26.142 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T11:16:26.370 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T11:16:26.386 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:26.411 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T11:16:26.428 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:26.439 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T11:16:26.477 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T11:16:26.767 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T11:16:26.793 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:26.798 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T11:16:26.825 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T11:16:26.833 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:26.864 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T11:16:27.120 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T11:16:27.146 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T11:16:27.176 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T11:16:27.186 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T11:16:27.203 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T11:16:27.242 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T11:16:27.489 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T11:16:27.516 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T11:16:27.555 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T11:16:27.570 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-py. 2024-04-25T11:16:27.598 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T11:16:27.636 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T11:16:27.900 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T11:16:27.927 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:27.957 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T11:16:27.966 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:27.984 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T11:16:28.023 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T11:16:28.320 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T11:16:28.335 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:28.369 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:28.452 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T11:16:28.480 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T11:16:28.519 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T11:16:28.796 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-toml. 2024-04-25T11:16:28.824 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T11:16:28.863 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T11:16:28.965 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T11:16:28.992 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:29.031 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:29.200 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T11:16:29.227 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T11:16:29.266 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T11:16:29.594 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T11:16:29.622 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T11:16:29.661 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T11:16:29.913 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T11:16:29.929 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T11:16:29.963 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T11:16:30.093 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph. 2024-04-25T11:16:30.108 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../068-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:30.151 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:30.429 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T11:16:30.455 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:30.495 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:30.618 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package radosgw. 2024-04-25T11:16:30.646 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../122-radosgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:30.676 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:30.824 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T11:16:30.850 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:30.890 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:31.301 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package cephadm. 2024-04-25T11:16:31.328 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../071-cephadm_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:31.368 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:31.846 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T11:16:31.862 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T11:16:31.872 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T11:16:31.890 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:31.912 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T11:16:31.928 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:32.242 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T11:16:32.265 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T11:16:32.292 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T11:16:32.331 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T11:16:32.526 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T11:16:32.652 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T11:16:32.915 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T11:16:33.256 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T11:16:33.286 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-babel. 2024-04-25T11:16:33.314 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T11:16:33.344 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T11:16:33.623 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T11:16:33.649 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T11:16:33.680 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T11:16:33.710 INFO:teuthology.orchestra.run.smithi134.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T11:16:33.778 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:33.992 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T11:16:34.007 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:34.042 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:34.278 INFO:teuthology.orchestra.run.smithi134.stdout:Adding system user cephadm....done 2024-04-25T11:16:34.379 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T11:16:34.393 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T11:16:34.406 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T11:16:34.437 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T11:16:34.669 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T11:16:34.715 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-routes. 2024-04-25T11:16:34.741 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T11:16:34.781 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T11:16:34.929 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T11:16:35.038 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T11:16:35.068 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T11:16:35.095 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:35.134 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:35.323 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T11:16:35.591 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T11:16:35.717 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T11:16:36.027 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T11:16:36.364 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T11:16:36.380 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T11:16:36.387 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T11:16:36.415 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T11:16:36.646 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T11:16:36.801 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T11:16:36.828 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T11:16:36.859 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T11:16:36.881 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T11:16:37.149 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T11:16:37.222 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T11:16:37.249 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T11:16:37.365 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T11:16:37.392 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T11:16:37.610 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T11:16:37.637 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T11:16:37.644 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T11:16:37.676 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T11:16:37.919 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T11:16:38.271 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:38.352 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T11:16:38.380 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:38.418 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:38.530 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T11:16:38.631 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T11:16:38.740 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T11:16:38.999 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T11:16:39.126 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T11:16:39.364 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T11:16:39.391 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T11:16:39.402 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T11:16:39.430 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T11:16:39.675 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T11:16:39.701 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T11:16:39.733 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T11:16:39.772 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T11:16:39.800 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T11:16:39.855 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T11:16:39.956 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T11:16:40.028 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T11:16:40.055 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T11:16:40.095 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T11:16:40.299 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T11:16:40.406 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T11:16:40.425 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T11:16:40.433 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T11:16:40.472 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T11:16:40.693 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T11:16:40.793 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T11:16:40.820 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T11:16:40.859 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T11:16:40.952 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T11:16:41.163 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T11:16:41.190 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T11:16:41.229 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T11:16:41.262 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T11:16:41.379 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T11:16:41.499 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T11:16:41.526 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T11:16:41.557 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T11:16:41.852 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T11:16:41.879 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T11:16:41.918 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T11:16:42.064 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T11:16:42.332 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T11:16:42.458 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T11:16:42.618 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T11:16:42.699 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T11:16:42.727 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:42.791 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:42.910 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T11:16:43.110 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T11:16:43.138 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T11:16:43.177 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T11:16:43.295 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T11:16:43.455 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T11:16:43.482 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T11:16:43.513 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T11:16:43.538 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T11:16:43.757 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package jq. 2024-04-25T11:16:43.785 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T11:16:43.824 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T11:16:43.906 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T11:16:44.077 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package socat. 2024-04-25T11:16:44.104 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T11:16:44.143 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T11:16:44.182 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T11:16:44.446 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T11:16:44.462 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T11:16:44.492 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T11:16:44.505 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T11:16:44.843 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T11:16:44.859 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-test. 2024-04-25T11:16:44.875 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:44.917 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:45.094 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T11:16:45.220 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:45.530 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T11:16:45.848 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T11:16:47.045 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T11:16:47.113 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T11:16:47.141 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:47.179 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:47.271 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:16:47.398 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T11:16:47.567 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T11:16:47.595 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:47.633 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:47.682 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T11:16:47.808 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T11:16:47.895 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T11:16:47.922 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T11:16:47.934 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T11:16:47.961 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T11:16:48.214 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T11:16:48.227 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T11:16:48.238 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:48.280 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:48.478 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T11:16:48.533 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T11:16:48.549 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T11:16:48.591 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:48.713 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:16:48.822 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T11:16:48.844 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T11:16:48.871 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:48.910 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:49.115 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T11:16:49.205 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T11:16:49.233 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T11:16:49.263 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T11:16:49.391 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T11:16:49.559 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T11:16:49.587 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T11:16:49.626 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T11:16:49.651 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T11:16:49.910 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T11:16:49.997 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T11:16:50.024 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T11:16:50.063 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T11:16:50.161 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T11:16:50.437 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T11:16:50.476 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package pkg-config. 2024-04-25T11:16:50.504 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T11:16:50.538 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T11:16:50.543 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T11:16:50.788 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T11:16:50.814 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T11:16:50.815 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T11:16:50.846 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T11:16:50.949 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T11:16:51.224 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T11:16:51.251 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T11:16:51.275 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T11:16:51.282 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T11:16:51.552 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T11:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T11:16:51.579 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T11:16:51.618 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T11:16:51.835 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T11:16:51.947 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T11:16:51.961 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T11:16:51.974 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T11:16:52.013 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T11:16:52.079 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T11:16:52.152 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T11:16:52.325 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T11:16:52.352 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T11:16:52.391 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T11:16:52.414 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T11:16:52.582 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T11:16:52.727 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-py. 2024-04-25T11:16:52.755 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T11:16:52.802 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T11:16:52.975 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T11:16:53.122 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T11:16:53.150 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T11:16:53.180 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T11:16:53.234 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T11:16:53.360 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T11:16:53.636 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:16:53.651 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T11:16:53.679 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T11:16:53.717 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T11:16:53.754 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T11:16:54.020 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-toml. 2024-04-25T11:16:54.049 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T11:16:54.055 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T11:16:54.086 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T11:16:54.331 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T11:16:54.350 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T11:16:54.378 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T11:16:54.408 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T11:16:54.633 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T11:16:54.753 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T11:16:54.781 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T11:16:54.819 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T11:16:54.852 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T11:16:55.089 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T11:16:55.116 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T11:16:55.120 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T11:16:55.155 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T11:16:55.363 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T11:16:55.606 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T11:16:55.809 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package radosgw. 2024-04-25T11:16:55.837 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../122-radosgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:55.876 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:55.983 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T11:16:56.276 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T11:16:56.603 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T11:16:56.705 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:56.820 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T11:16:56.822 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:56.836 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T11:16:56.878 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:56.957 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T11:16:57.183 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T11:16:57.293 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:57.419 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:57.450 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T11:16:57.537 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T11:16:57.576 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T11:16:57.788 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:57.836 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T11:16:57.914 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:58.049 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:58.161 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T11:16:58.256 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T11:16:58.257 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T11:16:58.590 INFO:teuthology.orchestra.run.smithi106.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T11:16:58.661 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:58.758 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:58.884 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:59.013 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:59.153 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:16:59.187 INFO:teuthology.orchestra.run.smithi106.stdout:Adding system user cephadm....done 2024-04-25T11:16:59.271 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T11:16:59.351 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T11:16:59.618 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T11:16:59.870 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T11:16:59.996 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T11:17:00.222 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T11:17:00.465 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T11:17:00.508 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:00.591 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T11:17:00.634 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:00.760 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:00.886 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:00.909 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T11:17:01.003 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:01.129 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:01.269 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T11:17:01.529 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T11:17:01.665 INFO:teuthology.orchestra.run.smithi134.stdout:Adding group ceph....done 2024-04-25T11:17:01.772 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T11:17:01.991 INFO:teuthology.orchestra.run.smithi134.stdout:Adding system user ceph....done 2024-04-25T11:17:02.023 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T11:17:02.151 INFO:teuthology.orchestra.run.smithi134.stdout:Setting system user ceph properties....done 2024-04-25T11:17:02.167 INFO:teuthology.orchestra.run.smithi134.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T11:17:02.283 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T11:17:02.309 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T11:17:02.542 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T11:17:02.638 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T11:17:02.802 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T11:17:03.086 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:03.131 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:03.344 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:17:03.345 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:17:03.354 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T11:17:03.463 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T11:17:03.564 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T11:17:03.816 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T11:17:03.824 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:03.941 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T11:17:03.967 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:04.201 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T11:17:04.437 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:17:04.437 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:17:04.590 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T11:17:04.623 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T11:17:04.679 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T11:17:04.788 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T11:17:04.944 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:05.140 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T11:17:05.265 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T11:17:05.525 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T11:17:05.537 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T11:17:05.784 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T11:17:06.005 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:06.094 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T11:17:06.220 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T11:17:06.237 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T11:17:06.237 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T11:17:06.707 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:06.905 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T11:17:06.925 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T11:17:06.925 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T11:17:07.148 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T11:17:07.274 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T11:17:07.408 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:07.416 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T11:17:07.687 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T11:17:07.687 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T11:17:07.725 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T11:17:08.110 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T11:17:08.186 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:08.303 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:08.361 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T11:17:08.454 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:08.667 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T11:17:08.667 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T11:17:08.738 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T11:17:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T11:17:09.155 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:09.307 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:09.332 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T11:17:09.432 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:09.584 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:09.675 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T11:17:09.926 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T11:17:09.936 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T11:17:10.037 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T11:17:10.052 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:10.181 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T11:17:10.361 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T11:17:10.671 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T11:17:11.341 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T11:17:11.760 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T11:17:12.007 INFO:teuthology.orchestra.run.smithi134.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T11:17:12.010 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T11:17:12.012 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:17:12.064 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-04-25T11:17:12.146 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T11:17:12.252 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-04-25T11:17:12.253 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-04-25T11:17:12.405 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T11:17:12.425 INFO:teuthology.orchestra.run.smithi134.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T11:17:12.425 INFO:teuthology.orchestra.run.smithi134.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T11:17:12.425 INFO:teuthology.orchestra.run.smithi134.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T11:17:12.425 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T11:17:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T11:17:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T11:17:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T11:17:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T11:17:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T11:17:12.427 INFO:teuthology.orchestra.run.smithi134.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T11:17:12.427 INFO:teuthology.orchestra.run.smithi134.stdout:The following additional packages will be installed: 2024-04-25T11:17:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T11:17:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T11:17:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T11:17:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T11:17:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T11:17:12.428 INFO:teuthology.orchestra.run.smithi134.stdout:Suggested packages: 2024-04-25T11:17:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T11:17:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T11:17:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T11:17:12.468 INFO:teuthology.orchestra.run.smithi134.stdout:The following NEW packages will be installed: 2024-04-25T11:17:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T11:17:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T11:17:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T11:17:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T11:17:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T11:17:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T11:17:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T11:17:12.470 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages will be upgraded: 2024-04-25T11:17:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T11:17:12.523 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T11:17:12.649 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T11:17:12.651 INFO:teuthology.orchestra.run.smithi134.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T11:17:12.651 INFO:teuthology.orchestra.run.smithi134.stdout:Need to get 55.7 MB of archives. 2024-04-25T11:17:12.651 INFO:teuthology.orchestra.run.smithi134.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T11:17:12.651 INFO:teuthology.orchestra.run.smithi134.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T11:17:12.934 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T11:17:13.185 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T11:17:13.281 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T11:17:13.298 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T11:17:13.423 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T11:17:13.424 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T11:17:13.425 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T11:17:13.436 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:17:13.453 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T11:17:13.455 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T11:17:13.458 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T11:17:13.461 INFO:teuthology.orchestra.run.smithi134.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T11:17:13.462 INFO:teuthology.orchestra.run.smithi134.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T11:17:13.628 INFO:teuthology.orchestra.run.smithi134.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T11:17:13.820 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T11:17:13.920 INFO:teuthology.orchestra.run.smithi134.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T11:17:13.930 INFO:teuthology.orchestra.run.smithi134.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T11:17:13.934 INFO:teuthology.orchestra.run.smithi134.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T11:17:14.019 INFO:teuthology.orchestra.run.smithi134.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T11:17:14.097 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T11:17:14.331 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T11:17:14.591 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T11:17:14.867 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T11:17:15.118 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T11:17:15.128 INFO:teuthology.orchestra.run.smithi134.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T11:17:15.132 INFO:teuthology.orchestra.run.smithi134.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T11:17:15.133 INFO:teuthology.orchestra.run.smithi134.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T11:17:15.139 INFO:teuthology.orchestra.run.smithi134.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T11:17:15.140 INFO:teuthology.orchestra.run.smithi134.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T11:17:15.150 INFO:teuthology.orchestra.run.smithi134.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T11:17:15.318 INFO:teuthology.orchestra.run.smithi134.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T11:17:15.385 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T11:17:15.511 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T11:17:15.796 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T11:17:15.905 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T11:17:15.908 INFO:teuthology.orchestra.run.smithi134.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T11:17:15.909 INFO:teuthology.orchestra.run.smithi134.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T11:17:16.088 INFO:teuthology.orchestra.run.smithi134.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T11:17:16.089 INFO:teuthology.orchestra.run.smithi134.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T11:17:16.127 INFO:teuthology.orchestra.run.smithi134.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T11:17:16.173 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T11:17:16.185 INFO:teuthology.orchestra.run.smithi134.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T11:17:16.192 INFO:teuthology.orchestra.run.smithi134.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T11:17:16.193 INFO:teuthology.orchestra.run.smithi134.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T11:17:16.194 INFO:teuthology.orchestra.run.smithi134.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T11:17:16.195 INFO:teuthology.orchestra.run.smithi134.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T11:17:16.405 INFO:teuthology.orchestra.run.smithi134.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T11:17:16.499 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T11:17:16.767 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T11:17:16.817 INFO:teuthology.orchestra.run.smithi134.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T11:17:16.893 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T11:17:17.019 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T11:17:17.107 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T11:17:17.345 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T11:17:17.463 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T11:17:17.610 INFO:teuthology.orchestra.run.smithi134.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T11:17:17.610 INFO:teuthology.orchestra.run.smithi134.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T11:17:17.657 INFO:teuthology.orchestra.run.smithi134.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T11:17:17.657 INFO:teuthology.orchestra.run.smithi134.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T11:17:17.657 INFO:teuthology.orchestra.run.smithi134.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T11:17:17.658 INFO:teuthology.orchestra.run.smithi134.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T11:17:17.688 INFO:teuthology.orchestra.run.smithi134.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T11:17:17.689 INFO:teuthology.orchestra.run.smithi134.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T11:17:17.831 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T11:17:18.091 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T11:17:18.093 INFO:teuthology.orchestra.run.smithi134.stdout:Preconfiguring packages ... 2024-04-25T11:17:18.217 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T11:17:18.312 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 55.7 MB in 5s (10.7 MB/s) 2024-04-25T11:17:18.476 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T11:17:18.616 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T11:17:18.622 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T11:17:18.627 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T11:17:18.768 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T11:17:18.953 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T11:17:19.279 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T11:17:19.546 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T11:17:19.547 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T11:17:19.750 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T11:17:20.018 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T11:17:20.294 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T11:17:20.501 INFO:teuthology.orchestra.run.smithi134.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T11:17:20.562 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T11:17:20.771 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package flex. 2024-04-25T11:17:20.866 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T11:17:20.871 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T11:17:20.906 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T11:17:20.919 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T11:17:21.207 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T11:17:21.535 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T11:17:21.543 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T11:17:21.694 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:21.750 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T11:17:21.820 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:21.938 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T11:17:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T11:17:22.266 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:22.392 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:22.518 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T11:17:22.722 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T11:17:22.727 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T11:17:22.794 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:22.849 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T11:17:22.920 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:23.046 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:23.239 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T11:17:23.240 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T11:17:23.292 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T11:17:23.406 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T11:17:23.758 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T11:17:23.806 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:23.966 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:24.140 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package bison. 2024-04-25T11:17:24.245 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T11:17:24.251 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T11:17:24.301 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T11:17:24.375 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:24.577 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:24.701 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T11:17:24.702 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T11:17:24.729 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T11:17:24.767 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T11:17:25.104 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package dump. 2024-04-25T11:17:25.131 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T11:17:25.171 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T11:17:25.566 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package indent. 2024-04-25T11:17:25.582 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:25.624 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T11:17:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:25.885 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T11:17:25.901 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T11:17:25.943 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T11:17:25.991 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:26.117 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:26.243 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:26.272 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T11:17:26.299 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T11:17:26.339 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T11:17:26.368 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:26.494 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:26.591 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T11:17:26.607 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T11:17:26.649 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T11:17:26.936 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T11:17:26.952 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T11:17:26.994 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T11:17:27.022 INFO:teuthology.orchestra.run.smithi106.stdout:Adding group ceph....done 2024-04-25T11:17:27.247 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T11:17:27.263 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T11:17:27.305 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T11:17:27.381 INFO:teuthology.orchestra.run.smithi106.stdout:Adding system user ceph....done 2024-04-25T11:17:27.516 INFO:teuthology.orchestra.run.smithi106.stdout:Setting system user ceph properties....done 2024-04-25T11:17:27.534 INFO:teuthology.orchestra.run.smithi106.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T11:17:27.540 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T11:17:27.557 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T11:17:27.599 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T11:17:27.666 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T11:17:27.902 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T11:17:27.918 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T11:17:27.952 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T11:17:27.993 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T11:17:28.453 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:28.654 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:17:28.654 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:17:29.316 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:29.476 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:29.672 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T11:17:29.700 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T11:17:29.741 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T11:17:29.950 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:17:29.950 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:17:30.008 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T11:17:30.025 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T11:17:30.066 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T11:17:30.369 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T11:17:30.394 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:30.397 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T11:17:30.436 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T11:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T11:17:30.764 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T11:17:30.791 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T11:17:30.830 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T11:17:31.166 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T11:17:31.171 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:31.195 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T11:17:31.233 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T11:17:31.391 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T11:17:31.391 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T11:17:31.586 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T11:17:31.614 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:31.652 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:31.864 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:31.997 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T11:17:32.025 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:32.072 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T11:17:32.090 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T11:17:32.090 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T11:17:32.416 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T11:17:32.432 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:32.482 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T11:17:32.557 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:32.802 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T11:17:32.830 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T11:17:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T11:17:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T11:17:32.868 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T11:17:33.317 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:33.372 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T11:17:33.400 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T11:17:33.465 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:33.581 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T11:17:33.703 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:33.916 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T11:17:33.916 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T11:17:33.968 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package modemmanager. 2024-04-25T11:17:33.996 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T11:17:34.035 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T11:17:34.379 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:34.733 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package network-manager. 2024-04-25T11:17:34.761 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T11:17:34.785 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:34.817 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T11:17:35.067 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:35.201 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T11:17:35.398 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ppp. 2024-04-25T11:17:35.426 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T11:17:35.571 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T11:17:35.672 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T11:17:35.685 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T11:17:35.833 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T11:17:35.970 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T11:17:35.997 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T11:17:36.036 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T11:17:36.273 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T11:17:36.301 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T11:17:36.339 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T11:17:36.625 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T11:17:36.654 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T11:17:36.691 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T11:17:36.959 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T11:17:36.970 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T11:17:36.998 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:37.037 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:37.604 INFO:teuthology.orchestra.run.smithi106.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T11:17:37.607 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T11:17:37.625 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T11:17:37.652 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T11:17:37.660 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2024-04-25T11:17:37.710 INFO:teuthology.orchestra.run.smithi134.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T11:17:37.739 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking postgresql-common (238) ... 2024-04-25T11:17:37.862 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2024-04-25T11:17:37.863 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2024-04-25T11:17:38.047 INFO:teuthology.orchestra.run.smithi106.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T11:17:38.048 INFO:teuthology.orchestra.run.smithi106.stdout:The following additional packages will be installed: 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout:Suggested packages: 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T11:17:38.087 INFO:teuthology.orchestra.run.smithi106.stdout:The following NEW packages will be installed: 2024-04-25T11:17:38.088 INFO:teuthology.orchestra.run.smithi106.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T11:17:38.088 INFO:teuthology.orchestra.run.smithi106.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T11:17:38.088 INFO:teuthology.orchestra.run.smithi106.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T11:17:38.088 INFO:teuthology.orchestra.run.smithi106.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T11:17:38.088 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T11:17:38.089 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T11:17:38.089 INFO:teuthology.orchestra.run.smithi106.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T11:17:38.089 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages will be upgraded: 2024-04-25T11:17:38.090 INFO:teuthology.orchestra.run.smithi106.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T11:17:38.156 INFO:teuthology.orchestra.run.smithi106.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T11:17:38.156 INFO:teuthology.orchestra.run.smithi106.stdout:Need to get 55.7 MB of archives. 2024-04-25T11:17:38.156 INFO:teuthology.orchestra.run.smithi106.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T11:17:38.156 INFO:teuthology.orchestra.run.smithi106.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T11:17:38.206 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T11:17:38.234 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:38.276 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:38.364 INFO:teuthology.orchestra.run.smithi106.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T11:17:38.374 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T11:17:38.433 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T11:17:38.434 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T11:17:38.435 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T11:17:38.454 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T11:17:38.455 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T11:17:38.458 INFO:teuthology.orchestra.run.smithi106.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T11:17:38.460 INFO:teuthology.orchestra.run.smithi106.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T11:17:38.461 INFO:teuthology.orchestra.run.smithi106.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T11:17:38.463 INFO:teuthology.orchestra.run.smithi106.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T11:17:38.476 INFO:teuthology.orchestra.run.smithi106.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T11:17:38.476 INFO:teuthology.orchestra.run.smithi106.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T11:17:38.477 INFO:teuthology.orchestra.run.smithi106.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T11:17:38.509 INFO:teuthology.orchestra.run.smithi106.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T11:17:39.085 INFO:teuthology.orchestra.run.smithi106.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T11:17:39.088 INFO:teuthology.orchestra.run.smithi106.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T11:17:39.088 INFO:teuthology.orchestra.run.smithi106.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T11:17:39.092 INFO:teuthology.orchestra.run.smithi106.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T11:17:39.092 INFO:teuthology.orchestra.run.smithi106.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T11:17:39.098 INFO:teuthology.orchestra.run.smithi106.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T11:17:39.101 INFO:teuthology.orchestra.run.smithi106.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T11:17:39.111 INFO:teuthology.orchestra.run.smithi106.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T11:17:39.111 INFO:teuthology.orchestra.run.smithi106.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T11:17:39.141 INFO:teuthology.orchestra.run.smithi106.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T11:17:39.144 INFO:teuthology.orchestra.run.smithi106.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T11:17:39.171 INFO:teuthology.orchestra.run.smithi106.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T11:17:39.213 INFO:teuthology.orchestra.run.smithi106.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T11:17:39.218 INFO:teuthology.orchestra.run.smithi106.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T11:17:39.219 INFO:teuthology.orchestra.run.smithi106.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T11:17:39.220 INFO:teuthology.orchestra.run.smithi106.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T11:17:39.220 INFO:teuthology.orchestra.run.smithi106.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T11:17:39.235 INFO:teuthology.orchestra.run.smithi106.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T11:17:39.238 INFO:teuthology.orchestra.run.smithi106.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T11:17:39.525 INFO:teuthology.orchestra.run.smithi106.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T11:17:39.525 INFO:teuthology.orchestra.run.smithi106.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T11:17:39.525 INFO:teuthology.orchestra.run.smithi106.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T11:17:39.525 INFO:teuthology.orchestra.run.smithi106.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T11:17:39.560 INFO:teuthology.orchestra.run.smithi106.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T11:17:39.560 INFO:teuthology.orchestra.run.smithi106.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T11:17:39.575 INFO:teuthology.orchestra.run.smithi106.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T11:17:39.576 INFO:teuthology.orchestra.run.smithi106.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T11:17:39.984 INFO:teuthology.orchestra.run.smithi106.stdout:Preconfiguring packages ... 2024-04-25T11:17:40.108 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package postgresql. 2024-04-25T11:17:40.122 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T11:17:40.165 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 55.7 MB in 1s (37.7 MB/s) 2024-04-25T11:17:40.166 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking postgresql (14+238) ... 2024-04-25T11:17:40.385 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T11:17:40.402 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T11:17:40.435 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T11:17:40.495 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T11:17:40.501 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T11:17:40.647 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T11:17:40.699 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T11:17:40.737 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T11:17:41.056 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T11:17:41.084 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T11:17:41.123 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T11:17:41.285 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T11:17:41.409 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package tcl. 2024-04-25T11:17:41.437 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T11:17:41.479 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T11:17:41.828 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T11:17:41.856 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T11:17:41.895 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T11:17:42.186 INFO:teuthology.orchestra.run.smithi106.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T11:17:42.257 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T11:17:42.273 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T11:17:42.307 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T11:17:42.436 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package flex. 2024-04-25T11:17:42.537 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T11:17:42.543 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T11:17:42.584 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T11:17:42.693 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T11:17:42.722 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T11:17:42.768 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T11:17:43.031 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T11:17:43.083 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T11:17:43.240 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T11:17:43.256 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T11:17:43.366 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T11:17:43.492 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:43.619 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T11:17:43.667 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T11:17:43.719 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T11:17:43.837 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:44.160 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T11:17:44.165 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T11:17:44.272 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T11:17:44.532 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T11:17:44.609 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T11:17:44.689 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T11:17:44.820 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T11:17:44.957 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T11:17:44.957 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T11:17:45.131 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T11:17:45.429 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T11:17:45.496 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package bison. 2024-04-25T11:17:45.546 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T11:17:45.598 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T11:17:45.603 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T11:17:45.648 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T11:17:45.672 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T11:17:45.798 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T11:17:45.924 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T11:17:46.040 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T11:17:46.068 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T11:17:46.075 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T11:17:46.107 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T11:17:46.201 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T11:17:46.327 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T11:17:46.404 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package dump. 2024-04-25T11:17:46.432 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T11:17:46.462 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T11:17:46.707 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package indent. 2024-04-25T11:17:46.735 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:46.774 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T11:17:46.906 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T11:17:47.035 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T11:17:47.063 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T11:17:47.101 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T11:17:47.174 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T11:17:47.268 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T11:17:47.334 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T11:17:47.439 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T11:17:47.443 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T11:17:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T11:17:47.497 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T11:17:47.544 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T11:17:47.662 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T11:17:47.758 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T11:17:47.786 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T11:17:47.788 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T11:17:47.824 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T11:17:47.878 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T11:17:47.947 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T11:17:48.074 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T11:17:48.153 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T11:17:48.180 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T11:17:48.219 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T11:17:48.250 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T11:17:48.376 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T11:17:48.438 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T11:17:48.454 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T11:17:48.485 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T11:17:48.497 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T11:17:48.620 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T11:17:48.729 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T11:17:48.749 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T11:17:48.777 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T11:17:48.816 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T11:17:49.128 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T11:17:49.155 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T11:17:49.194 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T11:17:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:17:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T11:17:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:which means they are currently configured by ifupdown: 2024-04-25T11:17:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:- enp3s0f1 2024-04-25T11:17:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T11:17:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T11:17:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:17:49.326 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T11:17:49.622 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T11:17:49.965 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T11:17:50.481 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T11:17:50.666 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T11:17:50.817 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T11:17:50.877 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T11:17:50.905 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T11:17:50.938 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T11:17:50.976 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up postgresql-common (238) ... 2024-04-25T11:17:51.199 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T11:17:51.215 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T11:17:51.257 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T11:17:51.569 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T11:17:51.596 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T11:17:51.635 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T11:17:51.909 INFO:teuthology.orchestra.run.smithi134.stdout:Adding user postgres to group ssl-cert 2024-04-25T11:17:51.947 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T11:17:51.975 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T11:17:52.013 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T11:17:52.175 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:17:52.175 INFO:teuthology.orchestra.run.smithi134.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T11:17:52.291 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T11:17:52.319 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T11:17:52.352 INFO:teuthology.orchestra.run.smithi134.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T11:17:52.352 INFO:teuthology.orchestra.run.smithi134.stdout:Removing obsolete dictionary files: 2024-04-25T11:17:52.357 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T11:17:52.694 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T11:17:52.721 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:52.760 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:52.922 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T11:17:53.096 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T11:17:53.125 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:53.163 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T11:17:53.303 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:53.441 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T11:17:53.457 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:53.490 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T11:17:53.623 INFO:teuthology.orchestra.run.smithi134.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T11:17:53.641 INFO:teuthology.orchestra.run.smithi134.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout:This user must also own the server process. 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout:The default text search configuration will be set to "english". 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout:Data page checksums are disabled. 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:17:53.680 INFO:teuthology.orchestra.run.smithi134.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T11:17:53.681 INFO:teuthology.orchestra.run.smithi134.stdout:creating subdirectories ... ok 2024-04-25T11:17:53.681 INFO:teuthology.orchestra.run.smithi134.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi134.stdout:selecting default max_connections ... 100 2024-04-25T11:17:53.717 INFO:teuthology.orchestra.run.smithi134.stdout:selecting default shared_buffers ... 128MB 2024-04-25T11:17:53.724 INFO:teuthology.orchestra.run.smithi134.stdout:selecting default time zone ... Etc/UTC 2024-04-25T11:17:53.724 INFO:teuthology.orchestra.run.smithi134.stdout:creating configuration files ... ok 2024-04-25T11:17:53.734 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T11:17:53.762 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T11:17:53.801 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T11:17:53.837 INFO:teuthology.orchestra.run.smithi134.stdout:running bootstrap script ... ok 2024-04-25T11:17:54.322 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T11:17:54.350 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T11:17:54.388 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T11:17:54.522 INFO:teuthology.orchestra.run.smithi134.stdout:performing post-bootstrap initialization ... ok 2024-04-25T11:17:54.749 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package modemmanager. 2024-04-25T11:17:54.777 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T11:17:54.816 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T11:17:55.148 INFO:teuthology.orchestra.run.smithi134.stdout:syncing data to disk ... ok 2024-04-25T11:17:55.389 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package network-manager. 2024-04-25T11:17:55.417 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T11:17:55.456 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T11:17:55.521 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T11:17:56.055 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ppp. 2024-04-25T11:17:56.083 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T11:17:56.352 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T11:17:56.659 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T11:17:56.687 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T11:17:56.725 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T11:17:56.962 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T11:17:56.978 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T11:17:57.011 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T11:17:57.264 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T11:17:57.292 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T11:17:57.323 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T11:17:57.626 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T11:17:57.654 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:57.693 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:58.306 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T11:17:58.334 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T11:17:58.369 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T11:17:58.392 INFO:teuthology.orchestra.run.smithi106.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T11:17:58.420 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking postgresql-common (238) ... 2024-04-25T11:17:58.478 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T11:17:58.605 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up postgresql (14+238) ... 2024-04-25T11:17:58.730 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T11:17:58.856 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T11:17:58.880 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T11:17:58.908 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T11:17:58.949 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:17:59.109 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T11:17:59.109 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T11:17:59.667 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T11:17:59.784 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T11:18:00.103 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T11:18:00.271 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T11:18:00.299 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package postgresql. 2024-04-25T11:18:00.316 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T11:18:00.357 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking postgresql (14+238) ... 2024-04-25T11:18:00.601 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T11:18:00.617 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T11:18:00.659 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T11:18:00.903 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T11:18:00.932 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T11:18:00.970 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T11:18:01.077 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T11:18:01.189 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T11:18:01.264 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T11:18:01.292 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T11:18:01.330 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T11:18:01.625 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package tcl. 2024-04-25T11:18:01.654 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T11:18:01.689 INFO:teuthology.orchestra.run.smithi134.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T11:18:01.696 DEBUG:teuthology.parallel:result is None 2024-04-25T11:18:01.704 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T11:18:02.061 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T11:18:02.091 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T11:18:02.128 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T11:18:02.482 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T11:18:02.498 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T11:18:02.540 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T11:18:02.918 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T11:18:02.946 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T11:18:02.993 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T11:18:03.281 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T11:18:03.490 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T11:18:03.616 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T11:18:03.742 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:18:03.860 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T11:18:03.961 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T11:18:04.087 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:18:04.549 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T11:18:04.625 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T11:18:04.987 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T11:18:04.987 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T11:18:05.478 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T11:18:05.604 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T11:18:05.730 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T11:18:05.856 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T11:18:06.090 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T11:18:06.575 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T11:18:06.701 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T11:18:06.818 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T11:18:07.515 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T11:18:07.641 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T11:18:07.735 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T11:18:07.801 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T11:18:07.919 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T11:18:08.028 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T11:18:08.137 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T11:18:08.263 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T11:18:08.354 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T11:18:08.423 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T11:18:08.549 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T11:18:08.725 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T11:18:08.860 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T11:18:08.961 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T11:18:09.129 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T11:18:09.255 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T11:18:09.755 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:09.755 INFO:teuthology.orchestra.run.smithi106.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T11:18:09.755 INFO:teuthology.orchestra.run.smithi106.stdout:which means they are currently configured by ifupdown: 2024-04-25T11:18:09.755 INFO:teuthology.orchestra.run.smithi106.stdout:- enp3s0f1 2024-04-25T11:18:09.755 INFO:teuthology.orchestra.run.smithi106.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T11:18:09.755 INFO:teuthology.orchestra.run.smithi106.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T11:18:09.755 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:09.869 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T11:18:10.185 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T11:18:10.518 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T11:18:10.999 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T11:18:11.159 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T11:18:11.270 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T11:18:11.369 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up postgresql-common (238) ... 2024-04-25T11:18:12.568 INFO:teuthology.orchestra.run.smithi106.stdout:Adding user postgres to group ssl-cert 2024-04-25T11:18:12.795 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:12.795 INFO:teuthology.orchestra.run.smithi106.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T11:18:12.979 INFO:teuthology.orchestra.run.smithi106.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T11:18:12.979 INFO:teuthology.orchestra.run.smithi106.stdout:Removing obsolete dictionary files: 2024-04-25T11:18:13.491 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T11:18:13.846 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T11:18:14.153 INFO:teuthology.orchestra.run.smithi106.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T11:18:14.183 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T11:18:14.282 INFO:teuthology.orchestra.run.smithi106.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T11:18:14.282 INFO:teuthology.orchestra.run.smithi106.stdout:This user must also own the server process. 2024-04-25T11:18:14.282 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:14.282 INFO:teuthology.orchestra.run.smithi106.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T11:18:14.282 INFO:teuthology.orchestra.run.smithi106.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T11:18:14.282 INFO:teuthology.orchestra.run.smithi106.stdout:The default text search configuration will be set to "english". 2024-04-25T11:18:14.282 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:14.283 INFO:teuthology.orchestra.run.smithi106.stdout:Data page checksums are disabled. 2024-04-25T11:18:14.283 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:14.283 INFO:teuthology.orchestra.run.smithi106.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T11:18:14.283 INFO:teuthology.orchestra.run.smithi106.stdout:creating subdirectories ... ok 2024-04-25T11:18:14.283 INFO:teuthology.orchestra.run.smithi106.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T11:18:14.300 INFO:teuthology.orchestra.run.smithi106.stdout:selecting default max_connections ... 100 2024-04-25T11:18:14.321 INFO:teuthology.orchestra.run.smithi106.stdout:selecting default shared_buffers ... 128MB 2024-04-25T11:18:14.341 INFO:teuthology.orchestra.run.smithi106.stdout:selecting default time zone ... Etc/UTC 2024-04-25T11:18:14.343 INFO:teuthology.orchestra.run.smithi106.stdout:creating configuration files ... ok 2024-04-25T11:18:14.465 INFO:teuthology.orchestra.run.smithi106.stdout:running bootstrap script ... ok 2024-04-25T11:18:15.586 INFO:teuthology.orchestra.run.smithi106.stdout:performing post-bootstrap initialization ... ok 2024-04-25T11:18:16.290 INFO:teuthology.orchestra.run.smithi106.stdout:syncing data to disk ... ok 2024-04-25T11:18:16.676 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T11:18:19.423 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T11:18:19.565 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T11:18:19.674 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up postgresql (14+238) ... 2024-04-25T11:18:19.783 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T11:18:19.893 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T11:18:20.153 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T11:18:20.153 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T11:18:20.687 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T11:18:20.780 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T11:18:21.107 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T11:18:21.259 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T11:18:22.268 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T11:18:22.384 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T11:18:22.886 INFO:teuthology.orchestra.run.smithi106.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T11:18:22.892 DEBUG:teuthology.parallel:result is None 2024-04-25T11:18:22.892 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:18:23.265 DEBUG:teuthology.orchestra.run.smithi106:> dpkg-query -W -f '${Version}' ceph 2024-04-25T11:18:23.284 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:18:23.285 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:18:23.285 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291-g994e2f20-1jammy is installed. 2024-04-25T11:18:23.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:18:23.651 DEBUG:teuthology.orchestra.run.smithi134:> dpkg-query -W -f '${Version}' ceph 2024-04-25T11:18:23.671 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:18:23.671 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T11:18:23.671 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291-g994e2f20-1jammy is installed. 2024-04-25T11:18:23.673 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T11:18:23.673 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:23.673 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:18:23.691 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:23.691 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:18:23.731 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T11:18:23.731 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:23.731 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:18:23.747 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:18:23.802 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:23.802 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:18:23.815 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:18:23.869 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T11:18:23.870 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:23.870 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:18:23.882 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:18:23.939 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:23.939 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:18:23.953 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:18:24.008 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T11:18:24.009 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:24.009 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:18:24.021 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:18:24.073 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:24.074 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:18:24.086 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:18:24.145 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T11:18:24.243 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T11:18:24.243 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /var/log/ceph 2024-04-25T11:18:24.246 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2024-04-25T11:18:24.259 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T11:18:24.259 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:18:24.304 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:18:24.316 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T11:18:24.317 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:18:24.356 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:18:24.375 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T11:18:24.376 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T11:18:24.376 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T11:18:24.376 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:24.376 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T11:18:24.415 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T11:18:24.493 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:24.493 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T11:18:24.499 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T11:18:24.579 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T11:18:24.579 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T11:18:24.579 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-readahead}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673120', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi106.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDApDp15FJhIISdM89JUbqpyW3/JA5lfpUD9DI3r/nygW1U9UCccNZS89Rdm6RlS06064mYoyJt6pyNKYcMMJNI=', 'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH5LKZnKvuNKEJw8Zd9Hhfl9/xjgNBUgjsKyzzMjcJ4WOEBQ6PsWuqxm/RM5rbMWFwprEEuGh2HBCkQnY3CCJZM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_readahead']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T11:18:24.580 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:18:24.582 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:18:24.622 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:18:24.628 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:18:24.678 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:24.678 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:18:24.687 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:18:24.687 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-04-25T11:18:24.735 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:18:24.735 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:18:24.735 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-04-25T11:18:24.735 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:18:24.735 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 11:12:01.700053305 +0000 2024-04-25T11:18:24.736 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 11:12:01.448057320 +0000 2024-04-25T11:18:24.736 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 11:12:01.448057320 +0000 2024-04-25T11:18:24.736 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2024-04-25T11:18:24.736 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:18:24.789 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T11:18:24.789 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T11:18:24.789 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.00034689 s, 1.5 MB/s 2024-04-25T11:18:24.791 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:18:24.841 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-04-25T11:18:24.887 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:18:24.887 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:18:24.887 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-25T11:18:24.888 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:18:24.888 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 11:12:02.184045777 +0000 2024-04-25T11:18:24.888 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 11:12:01.928049673 +0000 2024-04-25T11:18:24.888 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 11:12:01.928049673 +0000 2024-04-25T11:18:24.888 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2024-04-25T11:18:24.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:18:24.941 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T11:18:24.941 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T11:18:24.941 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000311887 s, 1.6 MB/s 2024-04-25T11:18:24.942 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:18:24.993 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-04-25T11:18:25.039 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:18:25.039 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:18:25.039 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-04-25T11:18:25.039 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:18:25.040 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 11:12:02.636039027 +0000 2024-04-25T11:18:25.040 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 11:12:02.400042550 +0000 2024-04-25T11:18:25.040 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 11:12:02.400042550 +0000 2024-04-25T11:18:25.040 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2024-04-25T11:18:25.040 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:18:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T11:18:25.094 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T11:18:25.094 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000352493 s, 1.5 MB/s 2024-04-25T11:18:25.095 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:18:25.144 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-04-25T11:18:25.191 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:18:25.191 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:18:25.191 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-04-25T11:18:25.191 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:18:25.191 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 11:12:03.108031873 +0000 2024-04-25T11:18:25.191 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 11:12:02.860035682 +0000 2024-04-25T11:18:25.191 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 11:12:02.860035682 +0000 2024-04-25T11:18:25.192 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2024-04-25T11:18:25.192 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:18:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T11:18:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T11:18:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000379437 s, 1.3 MB/s 2024-04-25T11:18:25.247 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:18:25.297 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:25.297 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:25.297 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:18:25.303 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:18:25.303 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-04-25T11:18:25.350 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:18:25.350 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:18:25.350 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-04-25T11:18:25.351 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:18:25.351 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-25 11:11:52.120996528 +0000 2024-04-25T11:18:25.351 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-25 11:11:51.868998779 +0000 2024-04-25T11:18:25.351 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-25 11:11:51.868998779 +0000 2024-04-25T11:18:25.351 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-04-25T11:18:25.351 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:18:25.405 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-25T11:18:25.405 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-25T11:18:25.405 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00038645 s, 1.3 MB/s 2024-04-25T11:18:25.406 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:18:25.455 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-04-25T11:18:25.506 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:18:25.506 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:18:25.506 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-04-25T11:18:25.506 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:18:25.506 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-25 11:11:52.628991991 +0000 2024-04-25T11:18:25.506 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-25 11:11:52.336994599 +0000 2024-04-25T11:18:25.506 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-25 11:11:52.336994599 +0000 2024-04-25T11:18:25.506 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-04-25T11:18:25.506 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:18:25.561 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-25T11:18:25.561 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-25T11:18:25.561 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000392773 s, 1.3 MB/s 2024-04-25T11:18:25.562 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:18:25.611 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-04-25T11:18:25.658 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:18:25.658 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:18:25.658 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-04-25T11:18:25.658 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:18:25.658 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-25 11:11:53.120987596 +0000 2024-04-25T11:18:25.658 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-25 11:11:52.860989918 +0000 2024-04-25T11:18:25.658 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-25 11:11:52.860989918 +0000 2024-04-25T11:18:25.658 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-04-25T11:18:25.659 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:18:25.712 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-25T11:18:25.712 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-25T11:18:25.712 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000380276 s, 1.3 MB/s 2024-04-25T11:18:25.714 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:18:25.763 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-04-25T11:18:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:18:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:18:25.810 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-04-25T11:18:25.810 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:18:25.810 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-25 11:11:53.600983307 +0000 2024-04-25T11:18:25.810 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-25 11:11:53.336985666 +0000 2024-04-25T11:18:25.810 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-25 11:11:53.336985666 +0000 2024-04-25T11:18:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-04-25T11:18:25.810 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:18:25.868 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-25T11:18:25.868 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-25T11:18:25.868 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000393437 s, 1.3 MB/s 2024-04-25T11:18:25.869 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:18:25.919 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:25.920 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:18:25.920 INFO:tasks.ceph:Generating config... 2024-04-25T11:18:25.921 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T11:18:25.921 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T11:18:25.921 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T11:18:25.921 INFO:tasks.ceph:[client] pid file = 2024-04-25T11:18:25.921 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T11:18:25.921 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T11:18:25.921 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T11:18:25.921 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T11:18:25.922 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T11:18:25.923 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T11:18:25.923 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T11:18:25.924 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T11:18:25.924 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T11:18:25.924 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T11:18:25.924 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T11:18:25.990 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T11:18:26.003 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T11:18:26.070 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:18:26.080 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.106'), ('mon.b', '172.21.15.134'), ('mon.c', '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]')] 2024-04-25T11:18:26.080 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.106,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T11:18:26.081 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:26.081 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:18:26.127 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.106 --add b 172.21.15.134 --addv c '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:26.241 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:26.241 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: generated fsid 8efdb3a5-72cb-436f-9be5-cf4e40adf704 2024-04-25T11:18:26.241 INFO:teuthology.orchestra.run.smithi106.stdout:setting min_mon_release = quincy 2024-04-25T11:18:26.241 INFO:teuthology.orchestra.run.smithi106.stdout:epoch 0 2024-04-25T11:18:26.241 INFO:teuthology.orchestra.run.smithi106.stdout:fsid 8efdb3a5-72cb-436f-9be5-cf4e40adf704 2024-04-25T11:18:26.241 INFO:teuthology.orchestra.run.smithi106.stdout:last_changed 2024-04-25T11:18:26.244376+0000 2024-04-25T11:18:26.241 INFO:teuthology.orchestra.run.smithi106.stdout:created 2024-04-25T11:18:26.244376+0000 2024-04-25T11:18:26.241 INFO:teuthology.orchestra.run.smithi106.stdout:min_mon_release 17 (quincy) 2024-04-25T11:18:26.242 INFO:teuthology.orchestra.run.smithi106.stdout:election_strategy: 1 2024-04-25T11:18:26.242 INFO:teuthology.orchestra.run.smithi106.stdout:0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-25T11:18:26.242 INFO:teuthology.orchestra.run.smithi106.stdout:1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-04-25T11:18:26.242 INFO:teuthology.orchestra.run.smithi106.stdout:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.c 2024-04-25T11:18:26.242 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T11:18:26.247 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:18:26.252 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8efdb3a5-72cb-436f-9be5-cf4e40adf704... 2024-04-25T11:18:26.252 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:18:26.296 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.324 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.325 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:18:26.326 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = 8efdb3a5-72cb-436f-9be5-cf4e40adf704 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.327 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.328 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2024-04-25T11:18:26.329 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: bdev async discard = True 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: bdev enable discard = True 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore allocator = bitmap 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 1/20 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 1/20 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 4/10 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:18:26.330 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-25T11:18:26.331 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:18:26.332 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: mon down mkfs grace = 300 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2024-04-25T11:18:26.333 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = "" 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-25T11:18:26.334 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = True 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = True 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-25T11:18:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-04-25T11:18:26.336 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-04-25T11:18:26.337 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-04-25T11:18:26.338 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.106,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = 8efdb3a5-72cb-436f-9be5-cf4e40adf704 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-04-25T11:18:26.339 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-04-25T11:18:26.340 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-04-25T11:18:26.341 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:18:26.342 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.343 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: mon op complaint time = 120 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T11:18:26.344 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: client mount timeout = 600 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = "" 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-04-25T11:18:26.345 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds = 20 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds balancer = 20 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug frag = True 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = True 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: mds op complaint time = 180 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-04-25T11:18:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-04-25T11:18:26.348 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T11:18:26.348 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T11:18:26.472 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T11:18:26.472 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:26.472 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T11:18:26.477 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:26.477 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T11:18:26.522 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:18:26.523 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:26.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:18:26.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:18:26.590 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:26.590 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:26.639 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:18:26.639 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:26.639 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:18:26.639 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:18:26.662 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:26.662 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:26.710 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T11:18:26.710 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T11:18:26.710 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:18:26.784 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:18:26.791 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:18:26.865 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:18:26.878 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T11:18:26.878 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T11:18:26.952 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T11:18:26.958 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T11:18:26.970 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T11:18:27.077 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T11:18:27.083 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T11:18:27.095 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T11:18:27.169 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T11:18:27.176 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T11:18:27.189 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T11:18:27.303 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T11:18:27.310 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T11:18:27.320 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T11:18:27.321 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T11:18:27.388 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T11:18:27.403 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T11:18:27.464 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T11:18:27.478 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T11:18:27.544 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T11:18:27.559 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T11:18:27.619 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T11:18:27.633 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T11:18:27.633 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:18:27.633 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T11:18:27.647 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:27.647 INFO:tasks.ceph:role: osd.0 2024-04-25T11:18:27.647 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:18:27.647 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:18:27.710 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:18:27.711 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:18:28.161 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-25T11:18:28.163 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-25T11:18:28.164 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T11:18:28.213 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T11:18:28.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:28.225 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T11:18:28.225 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T11:18:28.279 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:28.279 INFO:tasks.ceph:role: osd.1 2024-04-25T11:18:28.279 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:18:28.279 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:18:28.341 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:18:28.341 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:18:28.341 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:18:28.341 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:18:28.342 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:18:28.342 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:18:28.342 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:18:28.342 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:18:28.342 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:18:28.342 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:18:28.815 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-25T11:18:28.816 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-25T11:18:28.817 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T11:18:28.941 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T11:18:28.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:28.953 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T11:18:28.953 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T11:18:29.007 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:29.007 INFO:tasks.ceph:role: osd.2 2024-04-25T11:18:29.007 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:18:29.007 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:18:29.070 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:18:29.071 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:18:29.553 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-25T11:18:29.555 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-25T11:18:29.555 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T11:18:29.700 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T11:18:29.711 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T11:18:29.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:29.712 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T11:18:29.767 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:29.767 INFO:tasks.ceph:role: osd.3 2024-04-25T11:18:29.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:18:29.767 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:18:29.830 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:18:29.830 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:18:29.830 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:18:29.830 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:18:29.831 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:18:29.831 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:18:29.831 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:18:29.831 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:18:29.831 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:18:29.831 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:18:30.275 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-25T11:18:30.277 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-25T11:18:30.278 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T11:18:30.364 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T11:18:30.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T11:18:30.374 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:30.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:30.523+0000 7f0e9e50f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T11:18:30.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:30.527+0000 7f0e9e50f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T11:18:30.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:30.527+0000 7f0e9e50f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:18:30.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:30.527+0000 7f0e9e50f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:18:30.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:30.527+0000 7f0e9e50f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:18:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:30.531+0000 7f0e9e50f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T11:18:33.524 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T11:18:33.538 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:33.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:33.651+0000 7f34774777c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T11:18:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:33.651+0000 7f34774777c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T11:18:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:33.651+0000 7f34774777c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:18:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:33.651+0000 7f34774777c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:18:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:33.651+0000 7f34774777c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:18:33.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:33.651+0000 7f34774777c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T11:18:36.634 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T11:18:36.647 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:36.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:36.759+0000 7fd126e4b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T11:18:36.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:36.759+0000 7fd126e4b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T11:18:36.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:36.759+0000 7fd126e4b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:18:36.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:36.759+0000 7fd126e4b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:18:36.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:36.759+0000 7fd126e4b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:18:36.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:36.759+0000 7fd126e4b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T11:18:39.756 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T11:18:39.769 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:39.883+0000 7f196b0e97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T11:18:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:39.883+0000 7f196b0e97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T11:18:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:39.883+0000 7f196b0e97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:18:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:39.883+0000 7f196b0e97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:18:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:39.883+0000 7f196b0e97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:18:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:18:39.883+0000 7f196b0e97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T11:18:42.893 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T11:18:42.906 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T11:18:42.923 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:42.923 INFO:tasks.ceph:role: osd.4 2024-04-25T11:18:42.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:18:42.923 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:18:42.984 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:18:42.985 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:18:43.447 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-25T11:18:43.449 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-25T11:18:43.449 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T11:18:43.542 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T11:18:43.553 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T11:18:43.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:43.554 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T11:18:43.607 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:43.607 INFO:tasks.ceph:role: osd.5 2024-04-25T11:18:43.607 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:18:43.608 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:18:43.669 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:18:43.669 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:18:43.669 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:18:43.669 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:18:43.669 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:18:43.669 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:18:43.669 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:18:43.670 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:18:43.670 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:18:43.670 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:18:44.138 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-25T11:18:44.139 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-25T11:18:44.139 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T11:18:44.283 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T11:18:44.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T11:18:44.296 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T11:18:44.349 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:44.349 INFO:tasks.ceph:role: osd.6 2024-04-25T11:18:44.350 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:18:44.350 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:18:44.413 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:18:44.906 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-25T11:18:44.908 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-25T11:18:44.908 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T11:18:45.065 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T11:18:45.077 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T11:18:45.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:45.078 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T11:18:45.133 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:18:45.133 INFO:tasks.ceph:role: osd.7 2024-04-25T11:18:45.133 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:18:45.134 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:18:45.197 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:18:45.623 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-25T11:18:45.625 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-25T11:18:45.625 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T11:18:45.721 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T11:18:45.732 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T11:18:45.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:45.733 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:45.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:45.888+0000 7f1506bb57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T11:18:45.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:45.888+0000 7f1506bb57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T11:18:45.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:45.888+0000 7f1506bb57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:18:45.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:45.888+0000 7f1506bb57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:18:45.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:45.888+0000 7f1506bb57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:18:45.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:45.892+0000 7f1506bb57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T11:18:48.865 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T11:18:48.879 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:48.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:48.992+0000 7f4609ee07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T11:18:48.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:48.992+0000 7f4609ee07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T11:18:48.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:48.992+0000 7f4609ee07c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:18:48.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:48.992+0000 7f4609ee07c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:18:48.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:48.992+0000 7f4609ee07c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:18:48.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:48.996+0000 7f4609ee07c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T11:18:51.972 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T11:18:51.985 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:52.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:52.099+0000 7fd4fd1127c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T11:18:52.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:52.099+0000 7fd4fd1127c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T11:18:52.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:52.099+0000 7fd4fd1127c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:18:52.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:52.099+0000 7fd4fd1127c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:18:52.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:52.099+0000 7fd4fd1127c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:18:52.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:52.099+0000 7fd4fd1127c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T11:18:54.582 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:18:54.584 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:18:55.097 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T11:18:55.111 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:18:55.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:55.219+0000 7f4f3c94a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T11:18:55.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:55.219+0000 7f4f3c94a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T11:18:55.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:55.219+0000 7f4f3c94a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:18:55.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:55.219+0000 7f4f3c94a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:18:55.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:55.219+0000 7f4f3c94a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:18:55.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:18:55.223+0000 7f4f3c94a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T11:18:58.208 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T11:18:58.221 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T11:18:58.221 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.221 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T11:18:58.234 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.235 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T11:18:58.289 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.289 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T11:18:58.341 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T11:18:58.394 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.394 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T11:18:58.450 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.450 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T11:18:58.507 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.507 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T11:18:58.561 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.562 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T11:18:58.576 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.576 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T11:18:58.628 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.628 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T11:18:58.685 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.685 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T11:18:58.744 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.744 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T11:18:58.800 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.800 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T11:18:58.856 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.857 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T11:18:58.912 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.913 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T11:18:58.918 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:18:58.918 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T11:18:58.967 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.967 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T11:18:58.973 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:18:58.973 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T11:18:59.021 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T11:18:59.022 DEBUG:teuthology.orchestra.run.smithi106:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:18:59.024 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:18:59.067 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.y] 2024-04-25T11:18:59.067 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQACPCpmEcHrLhAAoHf/nv5AgPZKvsAjrH/bsQ== 2024-04-25T11:18:59.067 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-25T11:18:59.067 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQACPCpmt6nyOBAAujM4DC6/QuUUc3m9T3BnWg== 2024-04-25T11:18:59.067 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-25T11:18:59.067 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADPCpmd/nIBBAAhxEcRyNRhKxwIP7z11osNg== 2024-04-25T11:18:59.067 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.0] 2024-04-25T11:18:59.067 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAGPCpm58GZHxAA4ukc6lqPm3tuB+nVCWvflg== 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.1] 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAJPCpmYocQJxAA4kmFs222wtG4o6KtWq8SHA== 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.2] 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAMPCpmjaCMLRAAhEZEfHFGfbp6h8I5Whkidw== 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.3] 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAPPCpmrpT0NBAATZTX0OHG9n4x5UpHopwzzg== 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.x] 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQACPCpmAIPHMxAACFuC5wW8a+Lhyl4gy+aOuQ== 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADPCpmzp9PChAA772mlqVvY6+7zwkaj1ImZg== 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADPCpm64lQEhAAVTJ1nmvnVVqHefhc5uonnw== 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.4] 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAVPCpmU2j/NBAAJ18g1a7XKUDnxp1K5GZYIQ== 2024-04-25T11:18:59.068 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.5] 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAYPCpmXGJYOxAAMaE55oFohnPXLoNNOiSiGw== 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.6] 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAcPCpmBVEdBhAAh/QAN9kWd9QpIivo1WQgKw== 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.7] 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAfPCpmCg5ZDRAApPy0RHPegmvJr1R7Vvu/ew== 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADPCpmT+VXFxAASImB5WSW6aNZRALgQGZkFQ== 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout:[client.2] 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADPCpm5uLmGxAAml2ZRg40PRw7bkxpM6f2gA== 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout:[client.1] 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADPCpm83KiIBAAdXUTXe8EL2RdKF4EsqCYiQ== 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout:[client.3] 2024-04-25T11:18:59.069 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADPCpmSX8cJRAAKVq+BB+WPau/0qh0oK7z8g== 2024-04-25T11:18:59.075 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.y] 2024-04-25T11:18:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQACPCpmEcHrLhAAoHf/nv5AgPZKvsAjrH/bsQ== 2024-04-25T11:18:59.075 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-04-25T11:18:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQACPCpmt6nyOBAAujM4DC6/QuUUc3m9T3BnWg== 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQADPCpmd/nIBBAAhxEcRyNRhKxwIP7z11osNg== 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAGPCpm58GZHxAA4ukc6lqPm3tuB+nVCWvflg== 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAJPCpmYocQJxAA4kmFs222wtG4o6KtWq8SHA== 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAMPCpmjaCMLRAAhEZEfHFGfbp6h8I5Whkidw== 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2024-04-25T11:18:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAPPCpmrpT0NBAATZTX0OHG9n4x5UpHopwzzg== 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQACPCpmAIPHMxAACFuC5wW8a+Lhyl4gy+aOuQ== 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQADPCpmzp9PChAA772mlqVvY6+7zwkaj1ImZg== 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQADPCpm64lQEhAAVTJ1nmvnVVqHefhc5uonnw== 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAVPCpmU2j/NBAAJ18g1a7XKUDnxp1K5GZYIQ== 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAYPCpmXGJYOxAAMaE55oFohnPXLoNNOiSiGw== 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.6] 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAcPCpmBVEdBhAAh/QAN9kWd9QpIivo1WQgKw== 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.7] 2024-04-25T11:18:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAfPCpmCg5ZDRAApPy0RHPegmvJr1R7Vvu/ew== 2024-04-25T11:18:59.078 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2024-04-25T11:18:59.078 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQADPCpmT+VXFxAASImB5WSW6aNZRALgQGZkFQ== 2024-04-25T11:18:59.078 INFO:teuthology.orchestra.run.smithi134.stdout:[client.2] 2024-04-25T11:18:59.078 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQADPCpm5uLmGxAAml2ZRg40PRw7bkxpM6f2gA== 2024-04-25T11:18:59.078 INFO:teuthology.orchestra.run.smithi134.stdout:[client.1] 2024-04-25T11:18:59.078 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQADPCpm83KiIBAAdXUTXe8EL2RdKF4EsqCYiQ== 2024-04-25T11:18:59.078 INFO:teuthology.orchestra.run.smithi134.stdout:[client.3] 2024-04-25T11:18:59.078 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQADPCpmSX8cJRAAKVq+BB+WPau/0qh0oK7z8g== 2024-04-25T11:18:59.082 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:18:59.116 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:18:59.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:18:59.207 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:18:59.282 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:18:59.289 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:18:59.365 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.369 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.440 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.447 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.519 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.524 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.596 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.601 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.671 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:18:59.674 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:18:59.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:18:59.743 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:18:59.819 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:18:59.821 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:18:59.898 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.901 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.973 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:18:59.975 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:19:00.051 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:19:00.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:19:00.127 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:19:00.129 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:19:00.197 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:19:00.200 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:19:00.273 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:19:00.275 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:19:00.348 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:19:00.350 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:19:00.422 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:19:00.424 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:19:00.496 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T11:19:00.496 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T11:19:00.506 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:19:00.624 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T11:19:00.635 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T11:19:00.648 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:19:00.778 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T11:19:00.788 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T11:19:00.842 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:19:00.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T11:19:00.980 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:19:00.983 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:19:01.029 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T11:19:01.029 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T11:19:01.029 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T11:19:01.032 INFO:tasks.ceph.mon.a:Started 2024-04-25T11:19:01.032 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T11:19:01.032 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T11:19:01.074 INFO:tasks.ceph.mon.b:Started 2024-04-25T11:19:01.074 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T11:19:01.074 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T11:19:01.076 INFO:tasks.ceph.mon.c:Started 2024-04-25T11:19:01.076 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T11:19:01.076 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T11:19:01.077 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T11:19:01.079 INFO:tasks.ceph.mgr.y:Started 2024-04-25T11:19:01.079 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T11:19:01.079 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T11:19:01.081 INFO:tasks.ceph.mgr.x:Started 2024-04-25T11:19:01.081 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T11:19:01.081 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T11:19:01.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f7189eb8640 1 Processor -- start 2024-04-25T11:19:01.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f7189eb8640 1 -- start start 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f7189eb8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 0x7f718406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f7189eb8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71841604c0 con 0x7f71840d1e20 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f7189eb8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f718406d510 con 0x7f71840ccac0 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f7189eb8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f71840b30b0 con 0x7f71840b2ca0 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f71837fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 0x7f718406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f7183fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f71840d1e20 0x7f71840cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39602/0 (socket says 172.21.15.106:39602) 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f7183fff640 1 -- 172.21.15.106:0/292355649 learned_addr learned my addr 172.21.15.106:0/292355649 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f71837fe640 1 -- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 msgr2=0x7f718406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:19:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f71837fe640 1 -- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 msgr2=0x7f718406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:19:01.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 0x7f718406cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:19:01.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.226+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 0x7f718406cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:19:01.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.430+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 0x7f718406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.430+0000 7f71837fe640 1 -- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 msgr2=0x7f718406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:19:01.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.430+0000 7f71837fe640 1 -- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 msgr2=0x7f718406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:19:01.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.430+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 0x7f718406cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:19:01.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.430+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 0x7f718406cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:19:01.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.490+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344692330 0 0) 0x7f71841604c0 con 0x7f71840d1e20 2024-04-25T11:19:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.490+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f716c003530 con 0x7f71840d1e20 2024-04-25T11:19:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1524344032 0 0) 0x7f716c003530 con 0x7f71840d1e20 2024-04-25T11:19:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 msgr2=0x7f718406cfd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:19:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 --2- 172.21.15.106:0/292355649 >> v2:172.21.15.106:3300/0 conn(0x7f71840b2ca0 0x7f718406cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 >> v1:172.21.15.134:6789/0 conn(0x7f71840ccac0 legacy=0x7f71840d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71840b3980 con 0x7f71840d1e20 2024-04-25T11:19:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f717c001a00 con 0x7f71840d1e20 2024-04-25T11:19:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 >> v1:172.21.15.106:6789/0 conn(0x7f71840d1e20 legacy=0x7f71840cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f716c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716c004810 0x7f716c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f716c005360 0x7f716c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f716c00c230 con 0x7f716c0039c0 2024-04-25T11:19:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f716c00c3e0 con 0x7f716c004810 2024-04-25T11:19:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f716c00c5e0 con 0x7f716c005360 2024-04-25T11:19:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f716c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f7182ffd640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716c004810 0x7f716c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.494+0000 7f7183fff640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f716c005360 0x7f716c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71837fe640 1 -- 172.21.15.106:0/292355649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f716c005360 msgr2=0x7f716c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:01.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f716c005360 0x7f716c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71837fe640 1 -- 172.21.15.106:0/292355649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716c004810 msgr2=0x7f716c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:01.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716c004810 0x7f716c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71837fe640 1 -- 172.21.15.106:0/292355649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c003530 con 0x7f716c0039c0 2024-04-25T11:19:01.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f7183fff640 1 -- 172.21.15.106:0/292355649 reap_dead start 2024-04-25T11:19:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71837fe640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f716c003df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71840cff90 tx=0x7f717401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b0bbbf332ab5d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71740ae020 con 0x7f716c0039c0 2024-04-25T11:19:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71740a6630 con 0x7f716c0039c0 2024-04-25T11:19:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.498+0000 7f71827fc640 1 -- 172.21.15.106:0/292355649 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717400d660 con 0x7f716c0039c0 2024-04-25T11:19:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.502+0000 7f7189eb8640 1 -- 172.21.15.106:0/292355649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 msgr2=0x7f716c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.502+0000 7f7189eb8640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f716c003df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71840cff90 tx=0x7f717401c940 comp rx=0 tx=0).stop 2024-04-25T11:19:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.502+0000 7f7189eb8640 1 -- 172.21.15.106:0/292355649 shutdown_connections 2024-04-25T11:19:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.502+0000 7f7189eb8640 1 --2- 172.21.15.106:0/292355649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f716c003df0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.502+0000 7f7189eb8640 1 -- 172.21.15.106:0/292355649 >> 172.21.15.106:0/292355649 conn(0x7f71840b4020 msgr2=0x7f71840b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.502+0000 7f7189eb8640 1 -- 172.21.15.106:0/292355649 shutdown_connections 2024-04-25T11:19:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.502+0000 7f7189eb8640 1 -- 172.21.15.106:0/292355649 wait complete. 2024-04-25T11:19:01.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7189eb8640 1 Processor -- start 2024-04-25T11:19:01.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7189eb8640 1 -- start start 2024-04-25T11:19:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7189eb8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f7184154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7189eb8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71841552e0 0x7f71840b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7189eb8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7184156840 0x7f71840b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7189eb8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f71840cced0 con 0x7f716c0039c0 2024-04-25T11:19:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7189eb8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71840cfcc0 con 0x7f7184156840 2024-04-25T11:19:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7189eb8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f71840cd050 con 0x7f71841552e0 2024-04-25T11:19:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f71837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f7184154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f71837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f7184154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35636/0 (socket says 172.21.15.106:35636) 2024-04-25T11:19:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f71837fe640 1 -- 172.21.15.106:0/3918193689 learned_addr learned my addr 172.21.15.106:0/3918193689 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7182ffd640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71841552e0 0x7f71840b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.506+0000 7f7183fff640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7184156840 0x7f71840b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f71837fe640 1 -- 172.21.15.106:0/3918193689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71841552e0 msgr2=0x7f71840b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f71837fe640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71841552e0 0x7f71840b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f71837fe640 1 -- 172.21.15.106:0/3918193689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7184156840 msgr2=0x7f71840b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f71837fe640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7184156840 0x7f71840b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f71837fe640 1 -- 172.21.15.106:0/3918193689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71841550f0 con 0x7f716c0039c0 2024-04-25T11:19:01.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f71837fe640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f7184154bb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f717401c770 tx=0x7f716c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f716bfff640 1 -- 172.21.15.106:0/3918193689 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71740ab040 con 0x7f716c0039c0 2024-04-25T11:19:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f716bfff640 1 -- 172.21.15.106:0/3918193689 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71740b9070 con 0x7f716c0039c0 2024-04-25T11:19:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71841629e0 con 0x7f716c0039c0 2024-04-25T11:19:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f716bfff640 1 -- 172.21.15.106:0/3918193689 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717400d660 con 0x7f716c0039c0 2024-04-25T11:19:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7184155d30 con 0x7f716c0039c0 2024-04-25T11:19:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f716bfff640 1 -- 172.21.15.106:0/3918193689 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71740b1020 con 0x7f716c0039c0 2024-04-25T11:19:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.510+0000 7f716bfff640 1 -- 172.21.15.106:0/3918193689 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f71740beaa0 con 0x7f716c0039c0 2024-04-25T11:19:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.514+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7184050750 con 0x7f716c0039c0 2024-04-25T11:19:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.526+0000 7f716bfff640 1 -- 172.21.15.106:0/3918193689 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7174016690 con 0x7f716c0039c0 2024-04-25T11:19:01.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.590+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f71840b3240 con 0x7f716c0039c0 2024-04-25T11:19:01.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.594+0000 7f716bfff640 1 -- 172.21.15.106:0/3918193689 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f71740be460 con 0x7f716c0039c0 2024-04-25T11:19:01.593 INFO:teuthology.orchestra.run.smithi106.stderr:adjusted tunables profile to default 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 msgr2=0x7f7184154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f7184154bb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f717401c770 tx=0x7f716c005840 comp rx=0 tx=0).stop 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 shutdown_connections 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7184156840 0x7f71840b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71841552e0 0x7f71840b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 --2- 172.21.15.106:0/3918193689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f716c0039c0 0x7f7184154bb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 >> 172.21.15.106:0/3918193689 conn(0x7f71840b4020 msgr2=0x7f71840b5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 shutdown_connections 2024-04-25T11:19:01.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:01.598+0000 7f7189eb8640 1 -- 172.21.15.106:0/3918193689 wait complete. 2024-04-25T11:19:01.614 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T11:19:01.614 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T11:19:01.614 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:19:01.614 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T11:19:01.626 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:19:01.626 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T11:19:01.681 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:19:01.681 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T11:19:01.718 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:01.718+0000 7efcd4851180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:19:01.729 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:19:01.730 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T11:19:01.731 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:01.731+0000 7f0b96ae1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:19:01.782 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:19:01.782 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T11:19:01.796 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:19:01.796 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T11:19:01.848 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:19:01.848 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T11:19:01.900 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:19:01.900 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T11:19:01.952 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 73ae35d4-be76-494e-9a4c-4787c1736653 0 2024-04-25T11:19:02.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.147+0000 7fdcdb06b640 1 Processor -- start 2024-04-25T11:19:02.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.147+0000 7fdcdb06b640 1 -- start start 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcdb06b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdcd406dd10 0x7fdcd415c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcdb06b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd415d090 con 0x7fdcd4153230 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcdb06b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd40d17f0 con 0x7fdcd415cc80 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcdb06b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd4153640 con 0x7fdcd406dd10 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdcd406dd10 0x7fdcd415c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdcd406dd10 0x7fdcd415c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:52990/0 (socket says 172.21.15.134:52990) 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd9608640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fdcd415cc80 0x7fdcd4070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59516/0 (socket says 172.21.15.134:59516) 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd3fff640 1 -- 172.21.15.134:0/3428221526 learned_addr learned my addr 172.21.15.134:0/3428221526 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd3fff640 1 -- 172.21.15.134:0/3428221526 >> v1:172.21.15.134:6789/0 conn(0x7fdcd415cc80 legacy=0x7fdcd4070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd3fff640 1 -- 172.21.15.134:0/3428221526 >> v1:172.21.15.106:6789/0 conn(0x7fdcd4153230 legacy=0x7fdcd406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd37fe640 1 -- 172.21.15.134:0/3428221526 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336594243 0 0) 0x7fdcd40d17f0 con 0x7fdcd415cc80 2024-04-25T11:19:02.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd3fff640 1 -- 172.21.15.134:0/3428221526 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd415d600 con 0x7fdcd406dd10 2024-04-25T11:19:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd3fff640 1 --2- 172.21.15.134:0/3428221526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd406dd10 0x7fdcd415c740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800aa10 tx=0x7fdcc801d8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27c27c5903da28cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd37fe640 1 -- 172.21.15.134:0/3428221526 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80b3070 con 0x7fdcd406dd10 2024-04-25T11:19:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd37fe640 1 -- 172.21.15.134:0/3428221526 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc80a6980 con 0x7fdcd406dd10 2024-04-25T11:19:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.151+0000 7fdcd37fe640 1 -- 172.21.15.134:0/3428221526 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc800b040 con 0x7fdcd406dd10 2024-04-25T11:19:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- 172.21.15.134:0/3428221526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd406dd10 msgr2=0x7fdcd415c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 --2- 172.21.15.134:0/3428221526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd406dd10 0x7fdcd415c740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800aa10 tx=0x7fdcc801d8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- 172.21.15.134:0/3428221526 shutdown_connections 2024-04-25T11:19:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 --2- 172.21.15.134:0/3428221526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd406dd10 0x7fdcd415c740 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- 172.21.15.134:0/3428221526 >> 172.21.15.134:0/3428221526 conn(0x7fdcd4073300 msgr2=0x7fdcd4073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- 172.21.15.134:0/3428221526 shutdown_connections 2024-04-25T11:19:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- 172.21.15.134:0/3428221526 wait complete. 2024-04-25T11:19:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 Processor -- start 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- start start 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd4153230 0x7fdcd415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcd415cc80 0x7fdcd4154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd415f450 0x7fdcd4154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4068590 con 0x7fdcd415f450 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd40cfe20 con 0x7fdcd4153230 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcdb06b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd4068710 con 0x7fdcd415cc80 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcd8e07640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd4153230 0x7fdcd415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcd8e07640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd4153230 0x7fdcd415ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53002/0 (socket says 172.21.15.134:53002) 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcd415cc80 0x7fdcd4154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.155+0000 7fdcd8e07640 1 -- 172.21.15.134:0/2953001859 learned_addr learned my addr 172.21.15.134:0/2953001859 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd9608640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd415f450 0x7fdcd4154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd8e07640 1 -- 172.21.15.134:0/2953001859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcd415cc80 msgr2=0x7fdcd4154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd8e07640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcd415cc80 0x7fdcd4154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd8e07640 1 -- 172.21.15.134:0/2953001859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd415f450 msgr2=0x7fdcd4154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd8e07640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd415f450 0x7fdcd4154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd8e07640 1 -- 172.21.15.134:0/2953001859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd4155790 con 0x7fdcd4153230 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd9608640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd415f450 0x7fdcd4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd3fff640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcd415cc80 0x7fdcd4154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd8e07640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd4153230 0x7fdcd415ef10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c760 tx=0x7fdccc00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd17fa640 1 -- 172.21.15.134:0/2953001859 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc016020 con 0x7fdcd4153230 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd17fa640 1 -- 172.21.15.134:0/2953001859 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdccc006d60 con 0x7fdcd4153230 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd4155a70 con 0x7fdcd4153230 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd17fa640 1 -- 172.21.15.134:0/2953001859 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc003bc0 con 0x7fdcd4153230 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd4156030 con 0x7fdcd4153230 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd17fa640 1 -- 172.21.15.134:0/2953001859 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdccc00abe0 con 0x7fdcd4153230 2024-04-25T11:19:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcd17fa640 1 -- 172.21.15.134:0/2953001859 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fdccc0124b0 con 0x7fdcd4153230 2024-04-25T11:19:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.159+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd4050750 con 0x7fdcd4153230 2024-04-25T11:19:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.163+0000 7fdcd17fa640 1 -- 172.21.15.134:0/2953001859 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdccc020020 con 0x7fdcd4153230 2024-04-25T11:19:02.206 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:02.206+0000 7efcd4851180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:19:02.214 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:02.215+0000 7f0b96ae1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:19:02.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.231+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "73ae35d4-be76-494e-9a4c-4787c1736653", "id": 0} v 0) -- 0x7fdcd40d0b50 con 0x7fdcd4153230 2024-04-25T11:19:02.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcd17fa640 1 -- 172.21.15.134:0/2953001859 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73ae35d4-be76-494e-9a4c-4787c1736653", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fdccc0050d0 con 0x7fdcd4153230 2024-04-25T11:19:02.237 INFO:teuthology.orchestra.run.smithi134.stdout:0 2024-04-25T11:19:02.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd4153230 msgr2=0x7fdcd415ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd4153230 0x7fdcd415ef10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c760 tx=0x7fdccc00cc20 comp rx=0 tx=0).stop 2024-04-25T11:19:02.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 shutdown_connections 2024-04-25T11:19:02.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd415f450 0x7fdcd4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcd415cc80 0x7fdcd4154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 --2- 172.21.15.134:0/2953001859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcd4153230 0x7fdcd415ef10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 >> 172.21.15.134:0/2953001859 conn(0x7fdcd4073300 msgr2=0x7fdcd40afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 shutdown_connections 2024-04-25T11:19:02.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.239+0000 7fdcdb06b640 1 -- 172.21.15.134:0/2953001859 wait complete. 2024-04-25T11:19:02.257 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 02b209ec-e13e-428f-a041-ef9f203b4c52 1 2024-04-25T11:19:02.393 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:02.394+0000 7efcd4851180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:19:02.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a53de4640 1 Processor -- start 2024-04-25T11:19:02.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a53de4640 1 -- start start 2024-04-25T11:19:02.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a53de4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c1497f0 0x7f1a4c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a53de4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1a4c164020 0x7f1a4c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a53de4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c0b4c90 con 0x7f1a4c070c00 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a53de4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1a4c071010 con 0x7f1a4c164020 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a53de4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c0b5880 con 0x7f1a4c1497f0 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a51b80640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c1497f0 0x7f1a4c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a5137f640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f1a4c070c00 0x7f1a4c163910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59532/0 (socket says 172.21.15.134:59532) 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a5137f640 1 -- 172.21.15.134:0/3369517819 learned_addr learned my addr 172.21.15.134:0/3369517819 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a52381640 1 --2- 172.21.15.134:0/3369517819 >> v2:172.21.15.106:3300/0 conn(0x7f1a4c164020 0x7f1a4c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a50b7e640 1 -- 172.21.15.134:0/3369517819 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218141822 0 0) 0x7f1a4c0b4c90 con 0x7f1a4c070c00 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a50b7e640 1 -- 172.21.15.134:0/3369517819 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a34003530 con 0x7f1a4c070c00 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a51b80640 1 -- 172.21.15.134:0/3369517819 >> v2:172.21.15.106:3300/0 conn(0x7f1a4c164020 msgr2=0x7f1a4c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a51b80640 1 --2- 172.21.15.134:0/3369517819 >> v2:172.21.15.106:3300/0 conn(0x7f1a4c164020 0x7f1a4c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a51b80640 1 -- 172.21.15.134:0/3369517819 >> v1:172.21.15.134:6789/0 conn(0x7f1a4c070c00 legacy=0x7f1a4c163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.403+0000 7f1a51b80640 1 -- 172.21.15.134:0/3369517819 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c0b4f50 con 0x7f1a4c1497f0 2024-04-25T11:19:02.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a51b80640 1 --2- 172.21.15.134:0/3369517819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c1497f0 0x7f1a4c0706c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0b5a00 tx=0x7f1a4401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a216c4dc847e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a50b7e640 1 -- 172.21.15.134:0/3369517819 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a440aa600 con 0x7f1a4c1497f0 2024-04-25T11:19:02.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a50b7e640 1 -- 172.21.15.134:0/3369517819 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a440af070 con 0x7f1a4c1497f0 2024-04-25T11:19:02.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a50b7e640 1 -- 172.21.15.134:0/3369517819 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a440aa9c0 con 0x7f1a4c1497f0 2024-04-25T11:19:02.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a53de4640 1 -- 172.21.15.134:0/3369517819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c1497f0 msgr2=0x7f1a4c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a53de4640 1 --2- 172.21.15.134:0/3369517819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c1497f0 0x7f1a4c0706c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0b5a00 tx=0x7f1a4401c4a0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a53de4640 1 -- 172.21.15.134:0/3369517819 shutdown_connections 2024-04-25T11:19:02.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a53de4640 1 --2- 172.21.15.134:0/3369517819 >> v2:172.21.15.106:3300/0 conn(0x7f1a4c164020 0x7f1a4c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a53de4640 1 --2- 172.21.15.134:0/3369517819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c1497f0 0x7f1a4c0706c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a53de4640 1 -- 172.21.15.134:0/3369517819 >> 172.21.15.134:0/3369517819 conn(0x7f1a4c0d1ed0 msgr2=0x7f1a4c0d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a53de4640 1 -- 172.21.15.134:0/3369517819 shutdown_connections 2024-04-25T11:19:02.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.407+0000 7f1a53de4640 1 -- 172.21.15.134:0/3369517819 wait complete. 2024-04-25T11:19:02.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 Processor -- start 2024-04-25T11:19:02.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 -- start start 2024-04-25T11:19:02.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c070c00 0x7f1a4c1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a4c1497f0 0x7f1a4c142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c144050 0x7f1a4c144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c16ccb0 con 0x7f1a4c144050 2024-04-25T11:19:02.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c149c00 con 0x7f1a4c1497f0 2024-04-25T11:19:02.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c16ce30 con 0x7f1a4c070c00 2024-04-25T11:19:02.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a51b80640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c070c00 0x7f1a4c1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a5137f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a4c1497f0 0x7f1a4c142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a51b80640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c070c00 0x7f1a4c1608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33632/0 (socket says 172.21.15.134:33632) 2024-04-25T11:19:02.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a5137f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a4c1497f0 0x7f1a4c142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53016/0 (socket says 172.21.15.134:53016) 2024-04-25T11:19:02.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a51b80640 1 -- 172.21.15.134:0/1821493650 learned_addr learned my addr 172.21.15.134:0/1821493650 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:02.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a52381640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c144050 0x7f1a4c144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a5137f640 1 -- 172.21.15.134:0/1821493650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c070c00 msgr2=0x7f1a4c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a5137f640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c070c00 0x7f1a4c1608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a5137f640 1 -- 172.21.15.134:0/1821493650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c144050 msgr2=0x7f1a4c144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a5137f640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c144050 0x7f1a4c144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a5137f640 1 -- 172.21.15.134:0/1821493650 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c1f94c0 con 0x7f1a4c1497f0 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a51b80640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c070c00 0x7f1a4c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a5137f640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a4c1497f0 0x7f1a4c142250 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0b18d0 tx=0x7f1a40001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a4a7fc640 1 -- 172.21.15.134:0/1821493650 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40017070 con 0x7f1a4c1497f0 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a4a7fc640 1 -- 172.21.15.134:0/1821493650 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a40005cf0 con 0x7f1a4c1497f0 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c1449a0 con 0x7f1a4c1497f0 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a4a7fc640 1 -- 172.21.15.134:0/1821493650 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a400072f0 con 0x7f1a4c1497f0 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c0ce3d0 con 0x7f1a4c1497f0 2024-04-25T11:19:02.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a4a7fc640 1 -- 172.21.15.134:0/1821493650 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a4001a070 con 0x7f1a4c1497f0 2024-04-25T11:19:02.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.411+0000 7f1a4a7fc640 1 -- 172.21.15.134:0/1821493650 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f1a400127b0 con 0x7f1a4c1497f0 2024-04-25T11:19:02.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.415+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a4c142a80 con 0x7f1a4c1497f0 2024-04-25T11:19:02.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.415+0000 7f1a4a7fc640 1 -- 172.21.15.134:0/1821493650 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1a40021070 con 0x7f1a4c1497f0 2024-04-25T11:19:02.432 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:02.431+0000 7f0b96ae1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:19:02.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.487+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "02b209ec-e13e-428f-a041-ef9f203b4c52", "id": 1} v 0) -- 0x7f1a4c050750 con 0x7f1a4c1497f0 2024-04-25T11:19:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a4a7fc640 1 -- 172.21.15.134:0/1821493650 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "02b209ec-e13e-428f-a041-ef9f203b4c52", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f1a4c1449a0 con 0x7f1a4c1497f0 2024-04-25T11:19:02.498 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-25T11:19:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a4c1497f0 msgr2=0x7f1a4c142250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a4c1497f0 0x7f1a4c142250 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0b18d0 tx=0x7f1a40001e80 comp rx=0 tx=0).stop 2024-04-25T11:19:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 shutdown_connections 2024-04-25T11:19:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c144050 0x7f1a4c144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a4c1497f0 0x7f1a4c142250 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 --2- 172.21.15.134:0/1821493650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a4c070c00 0x7f1a4c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 >> 172.21.15.134:0/1821493650 conn(0x7f1a4c0d1ed0 msgr2=0x7f1a4c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 shutdown_connections 2024-04-25T11:19:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.499+0000 7f1a53de4640 1 -- 172.21.15.134:0/1821493650 wait complete. 2024-04-25T11:19:02.516 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 9f8f9520-a4aa-48db-b8fe-e3520a6b5f00 2 2024-04-25T11:19:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.663+0000 7f7969161640 1 Processor -- start 2024-04-25T11:19:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.663+0000 7f7969161640 1 -- start start 2024-04-25T11:19:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7969161640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7964161740 0x7f7964161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7969161640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964162090 0x7f7964068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7969161640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7964068ad0 0x7f7964068ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7969161640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f79640b1ed0 con 0x7f7964068ad0 2024-04-25T11:19:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7969161640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f796415ebd0 con 0x7f7964161740 2024-04-25T11:19:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7969161640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f796415f100 con 0x7f7964162090 2024-04-25T11:19:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7964161740 0x7f7964161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7964161740 0x7f7964161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53026/0 (socket says 172.21.15.134:53026) 2024-04-25T11:19:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 -- 172.21.15.134:0/1824502913 learned_addr learned my addr 172.21.15.134:0/1824502913 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f796259c640 1 --2- 172.21.15.134:0/1824502913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964162090 0x7f7964068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 -- 172.21.15.134:0/1824502913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964162090 msgr2=0x7f7964068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 --2- 172.21.15.134:0/1824502913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964162090 0x7f7964068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 -- 172.21.15.134:0/1824502913 >> v2:172.21.15.106:3300/0 conn(0x7f7964068ad0 msgr2=0x7f7964068ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 --2- 172.21.15.134:0/1824502913 >> v2:172.21.15.106:3300/0 conn(0x7f7964068ad0 0x7f7964068ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 -- 172.21.15.134:0/1824502913 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796415ee20 con 0x7f7964161740 2024-04-25T11:19:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7962d9d640 1 --2- 172.21.15.134:0/1824502913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964161740 0x7f7964161b50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f79640d22a0 tx=0x7f795401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae95fde8b84c1efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7961d9b640 1 -- 172.21.15.134:0/1824502913 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540ab640 con 0x7f7964161740 2024-04-25T11:19:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7961d9b640 1 -- 172.21.15.134:0/1824502913 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79540b0070 con 0x7f7964161740 2024-04-25T11:19:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.667+0000 7f7961d9b640 1 -- 172.21.15.134:0/1824502913 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540aba00 con 0x7f7964161740 2024-04-25T11:19:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 -- 172.21.15.134:0/1824502913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964161740 msgr2=0x7f7964161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 --2- 172.21.15.134:0/1824502913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964161740 0x7f7964161b50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f79640d22a0 tx=0x7f795401c480 comp rx=0 tx=0).stop 2024-04-25T11:19:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 -- 172.21.15.134:0/1824502913 shutdown_connections 2024-04-25T11:19:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 --2- 172.21.15.134:0/1824502913 >> v2:172.21.15.106:3300/0 conn(0x7f7964068ad0 0x7f7964068ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 --2- 172.21.15.134:0/1824502913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964162090 0x7f7964068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 --2- 172.21.15.134:0/1824502913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964161740 0x7f7964161b50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 -- 172.21.15.134:0/1824502913 >> 172.21.15.134:0/1824502913 conn(0x7f796409ba30 msgr2=0x7f79640b7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 -- 172.21.15.134:0/1824502913 shutdown_connections 2024-04-25T11:19:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 -- 172.21.15.134:0/1824502913 wait complete. 2024-04-25T11:19:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 Processor -- start 2024-04-25T11:19:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.671+0000 7f7969161640 1 -- start start 2024-04-25T11:19:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7964068ad0 0x7f796414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964161740 0x7f7964156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964162090 0x7f79641568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f796415dee0 con 0x7f7964068ad0 2024-04-25T11:19:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79641624a0 con 0x7f7964162090 2024-04-25T11:19:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f79640afb80 con 0x7f7964161740 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964161740 0x7f7964156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964162090 0x7f79641568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964161740 0x7f7964156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33652/0 (socket says 172.21.15.134:33652) 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 -- 172.21.15.134:0/4037472547 learned_addr learned my addr 172.21.15.134:0/4037472547 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7962d9d640 1 --2- 172.21.15.134:0/4037472547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7964068ad0 0x7f796414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 -- 172.21.15.134:0/4037472547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964162090 msgr2=0x7f79641568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 --2- 172.21.15.134:0/4037472547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964162090 0x7f79641568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 -- 172.21.15.134:0/4037472547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7964068ad0 msgr2=0x7f796414c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 --2- 172.21.15.134:0/4037472547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7964068ad0 0x7f796414c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 -- 172.21.15.134:0/4037472547 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796414e1b0 con 0x7f7964161740 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f796259c640 1 --2- 172.21.15.134:0/4037472547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964161740 0x7f7964156380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f795800e8d0 tx=0x7f795800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f794b7fe640 1 -- 172.21.15.134:0/4037472547 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958017070 con 0x7f7964161740 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f794b7fe640 1 -- 172.21.15.134:0/4037472547 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795800c940 con 0x7f7964161740 2024-04-25T11:19:02.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7964155ce0 con 0x7f7964161740 2024-04-25T11:19:02.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f794b7fe640 1 -- 172.21.15.134:0/4037472547 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795800ccf0 con 0x7f7964161740 2024-04-25T11:19:02.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f796414e450 con 0x7f7964161740 2024-04-25T11:19:02.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f794b7fe640 1 -- 172.21.15.134:0/4037472547 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f795801a070 con 0x7f7964161740 2024-04-25T11:19:02.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7964050750 con 0x7f7964161740 2024-04-25T11:19:02.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.675+0000 7f794b7fe640 1 -- 172.21.15.134:0/4037472547 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f7958013470 con 0x7f7964161740 2024-04-25T11:19:02.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.679+0000 7f794b7fe640 1 -- 172.21.15.134:0/4037472547 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7958006020 con 0x7f7964161740 2024-04-25T11:19:02.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.747+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9f8f9520-a4aa-48db-b8fe-e3520a6b5f00", "id": 2} v 0) -- 0x7f796409bff0 con 0x7f7964161740 2024-04-25T11:19:02.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.751+0000 7f794b7fe640 1 -- 172.21.15.134:0/4037472547 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f8f9520-a4aa-48db-b8fe-e3520a6b5f00", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7958006480 con 0x7f7964161740 2024-04-25T11:19:02.753 INFO:teuthology.orchestra.run.smithi134.stdout:2 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964161740 msgr2=0x7f7964156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 --2- 172.21.15.134:0/4037472547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964161740 0x7f7964156380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f795800e8d0 tx=0x7f795800ecd0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 shutdown_connections 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 --2- 172.21.15.134:0/4037472547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7964162090 0x7f79641568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 --2- 172.21.15.134:0/4037472547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7964161740 0x7f7964156380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 --2- 172.21.15.134:0/4037472547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7964068ad0 0x7f796414c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 >> 172.21.15.134:0/4037472547 conn(0x7f796409ba30 msgr2=0x7f79641541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 shutdown_connections 2024-04-25T11:19:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.755+0000 7f7969161640 1 -- 172.21.15.134:0/4037472547 wait complete. 2024-04-25T11:19:02.774 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new a1c3f77c-ad61-42ea-b742-611fb5c21cf9 3 2024-04-25T11:19:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f584d685640 1 Processor -- start 2024-04-25T11:19:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f584d685640 1 -- start start 2024-04-25T11:19:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f584d685640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58480d1b10 0x7f58480d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f584d685640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f584806e3b0 0x7f5848164940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f584d685640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58480d2850 con 0x7f584806dfa0 2024-04-25T11:19:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f584d685640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f58480d0f90 con 0x7f584806e3b0 2024-04-25T11:19:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f584d685640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f58480cf7c0 con 0x7f58480d1b10 2024-04-25T11:19:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f5846ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58480d1b10 0x7f58480d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f58467fc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f584806dfa0 0x7f5848164230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59534/0 (socket says 172.21.15.134:59534) 2024-04-25T11:19:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f58467fc640 1 -- 172.21.15.134:0/3395142347 learned_addr learned my addr 172.21.15.134:0/3395142347 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-25T11:19:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f58477fe640 1 --2- 172.21.15.134:0/3395142347 >> v2:172.21.15.106:3300/0 conn(0x7f584806e3b0 0x7f5848164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f5846ffd640 1 -- 172.21.15.134:0/3395142347 >> v2:172.21.15.106:3300/0 conn(0x7f584806e3b0 msgr2=0x7f5848164940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f5846ffd640 1 --2- 172.21.15.134:0/3395142347 >> v2:172.21.15.106:3300/0 conn(0x7f584806e3b0 0x7f5848164940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f5845ffb640 1 -- 172.21.15.134:0/3395142347 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629807355 0 0) 0x7f58480d2850 con 0x7f584806dfa0 2024-04-25T11:19:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f5846ffd640 1 -- 172.21.15.134:0/3395142347 >> v1:172.21.15.134:6789/0 conn(0x7f584806dfa0 legacy=0x7f5848164230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.915+0000 7f5846ffd640 1 -- 172.21.15.134:0/3395142347 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58480b8b60 con 0x7f58480d1b10 2024-04-25T11:19:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f5846ffd640 1 --2- 172.21.15.134:0/3395142347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58480d1b10 0x7f58480d1f20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f583c0097a0 tx=0x7f583c01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d590540498d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f5845ffb640 1 -- 172.21.15.134:0/3395142347 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c0b0070 con 0x7f58480d1b10 2024-04-25T11:19:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f5845ffb640 1 -- 172.21.15.134:0/3395142347 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f583c0ab600 con 0x7f58480d1b10 2024-04-25T11:19:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f5845ffb640 1 -- 172.21.15.134:0/3395142347 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c0ab9d0 con 0x7f58480d1b10 2024-04-25T11:19:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f584d685640 1 -- 172.21.15.134:0/3395142347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58480d1b10 msgr2=0x7f58480d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f584d685640 1 --2- 172.21.15.134:0/3395142347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58480d1b10 0x7f58480d1f20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f583c0097a0 tx=0x7f583c01c3f0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f584d685640 1 -- 172.21.15.134:0/3395142347 shutdown_connections 2024-04-25T11:19:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f584d685640 1 --2- 172.21.15.134:0/3395142347 >> v2:172.21.15.106:3300/0 conn(0x7f584806e3b0 0x7f5848164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f584d685640 1 --2- 172.21.15.134:0/3395142347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58480d1b10 0x7f58480d1f20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f584d685640 1 -- 172.21.15.134:0/3395142347 >> 172.21.15.134:0/3395142347 conn(0x7f58480cfff0 msgr2=0x7f58480d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f584d685640 1 -- 172.21.15.134:0/3395142347 shutdown_connections 2024-04-25T11:19:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.919+0000 7f584d685640 1 -- 172.21.15.134:0/3395142347 wait complete. 2024-04-25T11:19:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 Processor -- start 2024-04-25T11:19:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 -- start start 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f584806dfa0 0x7f584816a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f584806e3b0 0x7f584816aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58480d1b10 0x7f584815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58480cfc40 con 0x7f584806dfa0 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f584806e7c0 con 0x7f58480d1b10 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f58480cfdc0 con 0x7f584806e3b0 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58477fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58480d1b10 0x7f584815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58467fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f584806e3b0 0x7f584816aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58477fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58480d1b10 0x7f584815e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53034/0 (socket says 172.21.15.134:53034) 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58467fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f584806e3b0 0x7f584816aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33664/0 (socket says 172.21.15.134:33664) 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58477fe640 1 -- 172.21.15.134:0/3155307113 learned_addr learned my addr 172.21.15.134:0/3155307113 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:02.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f5846ffd640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f584806dfa0 0x7f584816a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58467fc640 1 -- 172.21.15.134:0/3155307113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58480d1b10 msgr2=0x7f584815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58467fc640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58480d1b10 0x7f584815e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58467fc640 1 -- 172.21.15.134:0/3155307113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f584806dfa0 msgr2=0x7f584816a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58467fc640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f584806dfa0 0x7f584816a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58467fc640 1 -- 172.21.15.134:0/3155307113 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58481628a0 con 0x7f584806e3b0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58477fe640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58480d1b10 0x7f584815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f5846ffd640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f584806dfa0 0x7f584816a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f58467fc640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f584806e3b0 0x7f584816aa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f583800c410 tx=0x7f583800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f582f7fe640 1 -- 172.21.15.134:0/3155307113 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58380074e0 con 0x7f584806e3b0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f582f7fe640 1 -- 172.21.15.134:0/3155307113 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58380047f0 con 0x7f584806e3b0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5848162b80 con 0x7f584806e3b0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f582f7fe640 1 -- 172.21.15.134:0/3155307113 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838007680 con 0x7f584806e3b0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f58481683c0 con 0x7f584806e3b0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f582f7fe640 1 -- 172.21.15.134:0/3155307113 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5838011070 con 0x7f584806e3b0 2024-04-25T11:19:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.923+0000 7f582f7fe640 1 -- 172.21.15.134:0/3155307113 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f5838013850 con 0x7f584806e3b0 2024-04-25T11:19:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.927+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848050750 con 0x7f584806e3b0 2024-04-25T11:19:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.927+0000 7f582f7fe640 1 -- 172.21.15.134:0/3155307113 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f58380133c0 con 0x7f584806e3b0 2024-04-25T11:19:02.945 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:02.946+0000 7efcd4851180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:19:02.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.991+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a1c3f77c-ad61-42ea-b742-611fb5c21cf9", "id": 3} v 0) -- 0x7f5848162d50 con 0x7f584806e3b0 2024-04-25T11:19:02.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f582f7fe640 1 -- 172.21.15.134:0/3155307113 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a1c3f77c-ad61-42ea-b742-611fb5c21cf9", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f5838013af0 con 0x7f584806e3b0 2024-04-25T11:19:02.998 INFO:teuthology.orchestra.run.smithi134.stdout:3 2024-04-25T11:19:02.999 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:03.002+0000 7efcd4851180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:19:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f584806e3b0 msgr2=0x7f584816aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f584806e3b0 0x7f584816aa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f583800c410 tx=0x7f583800c8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 shutdown_connections 2024-04-25T11:19:03.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58480d1b10 0x7f584815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f584806e3b0 0x7f584816aa80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 --2- 172.21.15.134:0/3155307113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f584806dfa0 0x7f584816a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 >> 172.21.15.134:0/3155307113 conn(0x7f58480cfff0 msgr2=0x7f58480725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 shutdown_connections 2024-04-25T11:19:03.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:02.999+0000 7f584d685640 1 -- 172.21.15.134:0/3155307113 wait complete. 2024-04-25T11:19:03.002 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.003+0000 7f0b96ae1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:19:03.017 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new c1985b6a-49ee-4e63-af0e-5c5ebf138b90 4 2024-04-25T11:19:03.051 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:03.054+0000 7efcd4851180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:19:03.059 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.059+0000 7f0b96ae1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:19:03.114 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.115+0000 7f0b96ae1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:19:03.127 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:03.130+0000 7efcd4851180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:19:03.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.163+0000 7f848e85d640 1 Processor -- start 2024-04-25T11:19:03.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.163+0000 7f848e85d640 1 -- start start 2024-04-25T11:19:03.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f848e85d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806cd00 0x7f84880ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f848e85d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84880b2cf0 con 0x7f8488161280 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f848e85d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84880cdff0 con 0x7f848806d110 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f848e85d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8488161690 con 0x7f848806cd00 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f84877fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806cd00 0x7f84880ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f848cdfa640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f848806d110 0x7f84880cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59546/0 (socket says 172.21.15.134:59546) 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f848cdfa640 1 -- 172.21.15.134:0/1023098254 learned_addr learned my addr 172.21.15.134:0/1023098254 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f8486ffd640 1 -- 172.21.15.134:0/1023098254 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1226539417 0 0) 0x7f84880cdff0 con 0x7f848806d110 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f8486ffd640 1 -- 172.21.15.134:0/1023098254 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8470003530 con 0x7f848806d110 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f84877fe640 1 -- 172.21.15.134:0/1023098254 >> v1:172.21.15.134:6789/0 conn(0x7f848806d110 legacy=0x7f84880cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f84877fe640 1 -- 172.21.15.134:0/1023098254 >> v1:172.21.15.106:6789/0 conn(0x7f8488161280 legacy=0x7f848806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.167+0000 7f84877fe640 1 -- 172.21.15.134:0/1023098254 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84880b2a10 con 0x7f848806cd00 2024-04-25T11:19:03.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f84877fe640 1 --2- 172.21.15.134:0/1023098254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806cd00 0x7f84880ce4d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f847c0097f0 tx=0x7f847c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64ed1ee5835c0e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f8486ffd640 1 -- 172.21.15.134:0/1023098254 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0b0070 con 0x7f848806cd00 2024-04-25T11:19:03.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f8486ffd640 1 -- 172.21.15.134:0/1023098254 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847c0ab600 con 0x7f848806cd00 2024-04-25T11:19:03.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f8486ffd640 1 -- 172.21.15.134:0/1023098254 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0ab9d0 con 0x7f848806cd00 2024-04-25T11:19:03.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 -- 172.21.15.134:0/1023098254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806cd00 msgr2=0x7f84880ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 --2- 172.21.15.134:0/1023098254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806cd00 0x7f84880ce4d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f847c0097f0 tx=0x7f847c01c450 comp rx=0 tx=0).stop 2024-04-25T11:19:03.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 -- 172.21.15.134:0/1023098254 shutdown_connections 2024-04-25T11:19:03.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 --2- 172.21.15.134:0/1023098254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806cd00 0x7f84880ce4d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 -- 172.21.15.134:0/1023098254 >> 172.21.15.134:0/1023098254 conn(0x7f84880b3f40 msgr2=0x7f84880b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 -- 172.21.15.134:0/1023098254 shutdown_connections 2024-04-25T11:19:03.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 -- 172.21.15.134:0/1023098254 wait complete. 2024-04-25T11:19:03.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 Processor -- start 2024-04-25T11:19:03.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.171+0000 7f848e85d640 1 -- start start 2024-04-25T11:19:03.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848806cd00 0x7f8488157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806d110 0x7f848815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8488161280 0x7f84880cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f848816bbb0 con 0x7f848806cd00 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f848816ba30 con 0x7f8488161280 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f848816bd30 con 0x7f848806d110 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806d110 0x7f848815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848cdfa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8488161280 0x7f84880cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806d110 0x7f848815c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33678/0 (socket says 172.21.15.134:33678) 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848cdfa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8488161280 0x7f84880cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53050/0 (socket says 172.21.15.134:53050) 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 -- 172.21.15.134:0/2828695555 learned_addr learned my addr 172.21.15.134:0/2828695555 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f8487fff640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848806cd00 0x7f8488157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 -- 172.21.15.134:0/2828695555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8488161280 msgr2=0x7f84880cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8488161280 0x7f84880cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 -- 172.21.15.134:0/2828695555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848806cd00 msgr2=0x7f8488157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848806cd00 0x7f8488157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 -- 172.21.15.134:0/2828695555 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848815c9b0 con 0x7f848806d110 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848cdfa640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8488161280 0x7f84880cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f84877fe640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806d110 0x7f848815c470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f847c009740 tx=0x7f84880cf930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f8484ff9640 1 -- 172.21.15.134:0/2828695555 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0b0040 con 0x7f848806d110 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f8484ff9640 1 -- 172.21.15.134:0/2828695555 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847c0b9070 con 0x7f848806d110 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848815ce30 con 0x7f848806d110 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f8484ff9640 1 -- 172.21.15.134:0/2828695555 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0b4650 con 0x7f848806d110 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f848815d3f0 con 0x7f848806d110 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f8484ff9640 1 -- 172.21.15.134:0/2828695555 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f847c0b47f0 con 0x7f848806d110 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f8484ff9640 1 -- 172.21.15.134:0/2828695555 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f847c0bed80 con 0x7f848806d110 2024-04-25T11:19:03.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.175+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848815cb80 con 0x7f848806d110 2024-04-25T11:19:03.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.179+0000 7f8484ff9640 1 -- 172.21.15.134:0/2828695555 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f847c0160d0 con 0x7f848806d110 2024-04-25T11:19:03.179 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:03.182+0000 7efcd4851180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:19:03.196 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.195+0000 7f0b96ae1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:19:03.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.247+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c1985b6a-49ee-4e63-af0e-5c5ebf138b90", "id": 4} v 0) -- 0x7f8488050750 con 0x7f848806d110 2024-04-25T11:19:03.249 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.251+0000 7f0b96ae1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:19:03.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f8484ff9640 1 -- 172.21.15.134:0/2828695555 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c1985b6a-49ee-4e63-af0e-5c5ebf138b90", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f847c0be460 con 0x7f848806d110 2024-04-25T11:19:03.253 INFO:teuthology.orchestra.run.smithi134.stdout:4 2024-04-25T11:19:03.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806d110 msgr2=0x7f848815c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806d110 0x7f848815c470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f847c009740 tx=0x7f84880cf930 comp rx=0 tx=0).stop 2024-04-25T11:19:03.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 shutdown_connections 2024-04-25T11:19:03.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8488161280 0x7f84880cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f848806d110 0x7f848815c470 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 --2- 172.21.15.134:0/2828695555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848806cd00 0x7f8488157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 >> 172.21.15.134:0/2828695555 conn(0x7f84880b3f40 msgr2=0x7f8488070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 shutdown_connections 2024-04-25T11:19:03.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.255+0000 7f848e85d640 1 -- 172.21.15.134:0/2828695555 wait complete. 2024-04-25T11:19:03.256 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:03.258+0000 7efcd4851180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:19:03.272 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 7ae5b812-55fc-41b9-8dfc-34108b145fcd 5 2024-04-25T11:19:03.309 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:03.310+0000 7efcd4851180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:19:03.330 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.331+0000 7f0b96ae1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:19:03.387 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.387+0000 7f0b96ae1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:19:03.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.451+0000 7f592dd97640 1 Processor -- start 2024-04-25T11:19:03.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.451+0000 7f592dd97640 1 -- start start 2024-04-25T11:19:03.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f592dd97640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f59280d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f592dd97640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59280b74e0 0x7f5928168240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f592dd97640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59280d1520 con 0x7f59280d2380 2024-04-25T11:19:03.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f592dd97640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f59280d1aa0 con 0x7f59280b74e0 2024-04-25T11:19:03.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f592dd97640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f59280cf370 con 0x7f592809ca40 2024-04-25T11:19:03.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f5927fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59280b74e0 0x7f5928168240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f5927fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59280b74e0 0x7f5928168240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53052/0 (socket says 172.21.15.134:53052) 2024-04-25T11:19:03.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f59277fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f59280d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f5927fff640 1 -- 172.21.15.134:0/3308850324 learned_addr learned my addr 172.21.15.134:0/3308850324 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:03.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f5927fff640 1 -- 172.21.15.134:0/3308850324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 msgr2=0x7f59280d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:03.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f5927fff640 1 --2- 172.21.15.134:0/3308850324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f59280d1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f5927fff640 1 -- 172.21.15.134:0/3308850324 >> v1:172.21.15.106:6789/0 conn(0x7f59280d2380 legacy=0x7f59280b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:03.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.455+0000 7f5927fff640 1 -- 172.21.15.134:0/3308850324 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59280b8d80 con 0x7f59280b74e0 2024-04-25T11:19:03.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f5927fff640 1 --2- 172.21.15.134:0/3308850324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928168240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5920009e50 tx=0x7f592001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46d5add551b98dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f59267fc640 1 -- 172.21.15.134:0/3308850324 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59200a5750 con 0x7f59280b74e0 2024-04-25T11:19:03.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f59267fc640 1 -- 172.21.15.134:0/3308850324 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59200a58f0 con 0x7f59280b74e0 2024-04-25T11:19:03.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f59267fc640 1 -- 172.21.15.134:0/3308850324 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59200a5cc0 con 0x7f59280b74e0 2024-04-25T11:19:03.458 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:03.458+0000 7efcd4851180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:19:03.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 -- 172.21.15.134:0/3308850324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 msgr2=0x7f5928168240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 --2- 172.21.15.134:0/3308850324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928168240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5920009e50 tx=0x7f592001c9a0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 -- 172.21.15.134:0/3308850324 shutdown_connections 2024-04-25T11:19:03.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 --2- 172.21.15.134:0/3308850324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928168240 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 --2- 172.21.15.134:0/3308850324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f59280d1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 -- 172.21.15.134:0/3308850324 >> 172.21.15.134:0/3308850324 conn(0x7f59280cfff0 msgr2=0x7f59280d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 -- 172.21.15.134:0/3308850324 shutdown_connections 2024-04-25T11:19:03.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 -- 172.21.15.134:0/3308850324 wait complete. 2024-04-25T11:19:03.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 Processor -- start 2024-04-25T11:19:03.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.459+0000 7f592dd97640 1 -- start start 2024-04-25T11:19:03.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f5928160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59280cc0b0 0x7f5928156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f59280c6ea0 con 0x7f59280cc0b0 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59280c6d20 con 0x7f59280b74e0 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f59280b88a0 con 0x7f592809ca40 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f59277fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f5928160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5926ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f59277fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f5928160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33692/0 (socket says 172.21.15.134:33692) 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f59277fe640 1 -- 172.21.15.134:0/1920589379 learned_addr learned my addr 172.21.15.134:0/1920589379 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5926ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53068/0 (socket says 172.21.15.134:53068) 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5927fff640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59280cc0b0 0x7f5928156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5926ffd640 1 -- 172.21.15.134:0/1920589379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 msgr2=0x7f5928160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5926ffd640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f5928160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5926ffd640 1 -- 172.21.15.134:0/1920589379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59280cc0b0 msgr2=0x7f5928156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5926ffd640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59280cc0b0 0x7f5928156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5926ffd640 1 -- 172.21.15.134:0/1920589379 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5928159cc0 con 0x7f59280b74e0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f59277fe640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f5928160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5927fff640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59280cc0b0 0x7f5928156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f5926ffd640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928154a10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5910000f70 tx=0x7f591000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f591ffff640 1 -- 172.21.15.134:0/1920589379 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910017070 con 0x7f59280b74e0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f591ffff640 1 -- 172.21.15.134:0/1920589379 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59100052d0 con 0x7f59280b74e0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f591ffff640 1 -- 172.21.15.134:0/1920589379 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910003840 con 0x7f59280b74e0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5928159fa0 con 0x7f59280b74e0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592815a560 con 0x7f59280b74e0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f591ffff640 1 -- 172.21.15.134:0/1920589379 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f591001a070 con 0x7f59280b74e0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f591ffff640 1 -- 172.21.15.134:0/1920589379 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5910011ae0 con 0x7f59280b74e0 2024-04-25T11:19:03.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.463+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5928154f50 con 0x7f59280b74e0 2024-04-25T11:19:03.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.467+0000 7f591ffff640 1 -- 172.21.15.134:0/1920589379 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5910021020 con 0x7f59280b74e0 2024-04-25T11:19:03.520 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:03.522+0000 7efcd4851180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:19:03.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.535+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7ae5b812-55fc-41b9-8dfc-34108b145fcd", "id": 5} v 0) -- 0x7f5928050750 con 0x7f59280b74e0 2024-04-25T11:19:03.544 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.543+0000 7f0b96ae1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:19:03.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.543+0000 7f591ffff640 1 -- 172.21.15.134:0/1920589379 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ae5b812-55fc-41b9-8dfc-34108b145fcd", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5910011670 con 0x7f59280b74e0 2024-04-25T11:19:03.544 INFO:teuthology.orchestra.run.smithi134.stdout:5 2024-04-25T11:19:03.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 msgr2=0x7f5928154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928154a10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5910000f70 tx=0x7f591000c820 comp rx=0 tx=0).stop 2024-04-25T11:19:03.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 shutdown_connections 2024-04-25T11:19:03.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59280cc0b0 0x7f5928156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59280b74e0 0x7f5928154a10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 --2- 172.21.15.134:0/1920589379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f592809ca40 0x7f5928160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 >> 172.21.15.134:0/1920589379 conn(0x7f59280cfff0 msgr2=0x7f59281617f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 shutdown_connections 2024-04-25T11:19:03.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.547+0000 7f592dd97640 1 -- 172.21.15.134:0/1920589379 wait complete. 2024-04-25T11:19:03.563 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new b09bb1a2-485c-42f4-ac06-2f4895e19994 6 2024-04-25T11:19:03.607 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:03.607+0000 7f0b96ae1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:19:03.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.743+0000 7f3e97ae7640 1 Processor -- start 2024-04-25T11:19:03.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.743+0000 7f3e97ae7640 1 -- start start 2024-04-25T11:19:03.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e97ae7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e900afad0 0x7f3e900afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e97ae7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3e9015c290 0x7f3e9015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e97ae7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e90070760 con 0x7f3e900b56a0 2024-04-25T11:19:03.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e97ae7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3e90070a40 con 0x7f3e9015c290 2024-04-25T11:19:03.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e97ae7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e900b4ea0 con 0x7f3e900afad0 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95883640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e900afad0 0x7f3e900afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e96084640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3e9015c290 0x7f3e9015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95082640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f3e900b56a0 0x7f3e900b5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59548/0 (socket says 172.21.15.134:59548) 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95883640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e900afad0 0x7f3e900afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33702/0 (socket says 172.21.15.134:33702) 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95082640 1 -- 172.21.15.134:0/252579578 learned_addr learned my addr 172.21.15.134:0/252579578 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e96084640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3e9015c290 0x7f3e9015c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53080/0 (socket says 172.21.15.134:53080) 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95883640 1 -- 172.21.15.134:0/252579578 >> v2:172.21.15.134:3300/0 conn(0x7f3e9015c290 msgr2=0x7f3e9015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95883640 1 --2- 172.21.15.134:0/252579578 >> v2:172.21.15.134:3300/0 conn(0x7f3e9015c290 0x7f3e9015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95883640 1 -- 172.21.15.134:0/252579578 >> v1:172.21.15.134:6789/0 conn(0x7f3e900b56a0 legacy=0x7f3e900b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e94881640 1 -- 172.21.15.134:0/252579578 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141155634 0 0) 0x7f3e90070760 con 0x7f3e900b56a0 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95883640 1 -- 172.21.15.134:0/252579578 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e900b5180 con 0x7f3e900afad0 2024-04-25T11:19:03.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e96084640 1 --2- 172.21.15.134:0/252579578 >> v2:172.21.15.134:3300/0 conn(0x7f3e9015c290 0x7f3e9015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:03.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e95883640 1 --2- 172.21.15.134:0/252579578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e900afad0 0x7f3e900afee0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0097c0 tx=0x7f3e8c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fbd06b54740c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e94881640 1 -- 172.21.15.134:0/252579578 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0b0070 con 0x7f3e900afad0 2024-04-25T11:19:03.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e94881640 1 -- 172.21.15.134:0/252579578 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8c0ab600 con 0x7f3e900afad0 2024-04-25T11:19:03.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.747+0000 7f3e94881640 1 -- 172.21.15.134:0/252579578 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0ab9d0 con 0x7f3e900afad0 2024-04-25T11:19:03.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- 172.21.15.134:0/252579578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e900afad0 msgr2=0x7f3e900afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 --2- 172.21.15.134:0/252579578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e900afad0 0x7f3e900afee0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0097c0 tx=0x7f3e8c01c410 comp rx=0 tx=0).stop 2024-04-25T11:19:03.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- 172.21.15.134:0/252579578 shutdown_connections 2024-04-25T11:19:03.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 --2- 172.21.15.134:0/252579578 >> v2:172.21.15.134:3300/0 conn(0x7f3e9015c290 0x7f3e9015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 --2- 172.21.15.134:0/252579578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e900afad0 0x7f3e900afee0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- 172.21.15.134:0/252579578 >> 172.21.15.134:0/252579578 conn(0x7f3e900d1ed0 msgr2=0x7f3e900d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- 172.21.15.134:0/252579578 shutdown_connections 2024-04-25T11:19:03.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- 172.21.15.134:0/252579578 wait complete. 2024-04-25T11:19:03.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 Processor -- start 2024-04-25T11:19:03.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- start start 2024-04-25T11:19:03.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e900afad0 0x7f3e90157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e900b56a0 0x7f3e9015f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e9015c290 0x7f3e901fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e90166f10 con 0x7f3e900b56a0 2024-04-25T11:19:03.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9015d640 con 0x7f3e900afad0 2024-04-25T11:19:03.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e97ae7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e90167090 con 0x7f3e9015c290 2024-04-25T11:19:03.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e95883640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e900afad0 0x7f3e90157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e96084640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e9015c290 0x7f3e901fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e95883640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e900afad0 0x7f3e90157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53094/0 (socket says 172.21.15.134:53094) 2024-04-25T11:19:03.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.751+0000 7f3e95883640 1 -- 172.21.15.134:0/1439069896 learned_addr learned my addr 172.21.15.134:0/1439069896 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:03.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e95883640 1 -- 172.21.15.134:0/1439069896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e9015c290 msgr2=0x7f3e901fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e95883640 1 --2- 172.21.15.134:0/1439069896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e9015c290 0x7f3e901fd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e95883640 1 -- 172.21.15.134:0/1439069896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e900b56a0 msgr2=0x7f3e9015f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e95883640 1 --2- 172.21.15.134:0/1439069896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e900b56a0 0x7f3e9015f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e95883640 1 -- 172.21.15.134:0/1439069896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9015e250 con 0x7f3e900afad0 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e96084640 1 --2- 172.21.15.134:0/1439069896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e9015c290 0x7f3e901fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e95883640 1 --2- 172.21.15.134:0/1439069896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e900afad0 0x7f3e90157ba0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c01a040 tx=0x7f3e8c0191b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e867fc640 1 -- 172.21.15.134:0/1439069896 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0b8020 con 0x7f3e900afad0 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e867fc640 1 -- 172.21.15.134:0/1439069896 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8c0aa7d0 con 0x7f3e900afad0 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e90140170 con 0x7f3e900afad0 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e867fc640 1 -- 172.21.15.134:0/1439069896 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0b0040 con 0x7f3e900afad0 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e901fced0 con 0x7f3e900afad0 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e867fc640 1 -- 172.21.15.134:0/1439069896 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e8c0bb020 con 0x7f3e900afad0 2024-04-25T11:19:03.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e867fc640 1 -- 172.21.15.134:0/1439069896 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f3e8c0b4b40 con 0x7f3e900afad0 2024-04-25T11:19:03.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e90050750 con 0x7f3e900afad0 2024-04-25T11:19:03.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.755+0000 7f3e867fc640 1 -- 172.21.15.134:0/1439069896 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3e8c0c2030 con 0x7f3e900afad0 2024-04-25T11:19:03.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.819+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b09bb1a2-485c-42f4-ac06-2f4895e19994", "id": 6} v 0) -- 0x7f3e9015bea0 con 0x7f3e900afad0 2024-04-25T11:19:03.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.831+0000 7f3e867fc640 1 -- 172.21.15.134:0/1439069896 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b09bb1a2-485c-42f4-ac06-2f4895e19994", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3e9015e250 con 0x7f3e900afad0 2024-04-25T11:19:03.832 INFO:teuthology.orchestra.run.smithi134.stdout:6 2024-04-25T11:19:03.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e900afad0 msgr2=0x7f3e90157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 --2- 172.21.15.134:0/1439069896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e900afad0 0x7f3e90157ba0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c01a040 tx=0x7f3e8c0191b0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 shutdown_connections 2024-04-25T11:19:03.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 --2- 172.21.15.134:0/1439069896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e9015c290 0x7f3e901fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 --2- 172.21.15.134:0/1439069896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e900b56a0 0x7f3e9015f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 --2- 172.21.15.134:0/1439069896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e900afad0 0x7f3e90157ba0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 >> 172.21.15.134:0/1439069896 conn(0x7f3e900d1ed0 msgr2=0x7f3e900b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 shutdown_connections 2024-04-25T11:19:03.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:03.835+0000 7f3e97ae7640 1 -- 172.21.15.134:0/1439069896 wait complete. 2024-04-25T11:19:03.850 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 245cbf8d-6718-4404-be12-ac042327c299 7 2024-04-25T11:19:04.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.067+0000 7fdaf6e26640 1 Processor -- start 2024-04-25T11:19:04.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.067+0000 7fdaf6e26640 1 -- start start 2024-04-25T11:19:04.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf6e26640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf00b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf6e26640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf00b6080 con 0x7fdaf00b0b60 2024-04-25T11:19:04.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf6e26640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf00af390 con 0x7fdaf015fdc0 2024-04-25T11:19:04.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf6e26640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf0160c00 con 0x7fdaf00b5730 2024-04-25T11:19:04.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf4bc2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf00b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaeffff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fdaf015fdc0 0x7fdaf01601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59562/0 (socket says 172.21.15.134:59562) 2024-04-25T11:19:04.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaeffff640 1 -- 172.21.15.134:0/2358999841 learned_addr learned my addr 172.21.15.134:0/2358999841 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-25T11:19:04.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaef7fe640 1 -- 172.21.15.134:0/2358999841 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775223386 0 0) 0x7fdaf00af390 con 0x7fdaf015fdc0 2024-04-25T11:19:04.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf4bc2640 1 -- 172.21.15.134:0/2358999841 >> v1:172.21.15.134:6789/0 conn(0x7fdaf015fdc0 legacy=0x7fdaf01601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf4bc2640 1 -- 172.21.15.134:0/2358999841 >> v1:172.21.15.106:6789/0 conn(0x7fdaf00b0b60 legacy=0x7fdaf0169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:04.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf4bc2640 1 -- 172.21.15.134:0/2358999841 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf0160ee0 con 0x7fdaf00b5730 2024-04-25T11:19:04.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.071+0000 7fdaf4bc2640 1 --2- 172.21.15.134:0/2358999841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf00b5b40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0097f0 tx=0x7fdadc01c290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=752cdb20531ab955 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:04.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaef7fe640 1 -- 172.21.15.134:0/2358999841 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0b0070 con 0x7fdaf00b5730 2024-04-25T11:19:04.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaef7fe640 1 -- 172.21.15.134:0/2358999841 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc0ab600 con 0x7fdaf00b5730 2024-04-25T11:19:04.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaef7fe640 1 -- 172.21.15.134:0/2358999841 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0ab9d0 con 0x7fdaf00b5730 2024-04-25T11:19:04.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 -- 172.21.15.134:0/2358999841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 msgr2=0x7fdaf00b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 --2- 172.21.15.134:0/2358999841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf00b5b40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0097f0 tx=0x7fdadc01c290 comp rx=0 tx=0).stop 2024-04-25T11:19:04.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 -- 172.21.15.134:0/2358999841 shutdown_connections 2024-04-25T11:19:04.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 --2- 172.21.15.134:0/2358999841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf00b5b40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 -- 172.21.15.134:0/2358999841 >> 172.21.15.134:0/2358999841 conn(0x7fdaf009ba30 msgr2=0x7fdaf00b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:04.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 -- 172.21.15.134:0/2358999841 shutdown_connections 2024-04-25T11:19:04.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 -- 172.21.15.134:0/2358999841 wait complete. 2024-04-25T11:19:04.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 Processor -- start 2024-04-25T11:19:04.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.075+0000 7fdaf6e26640 1 -- start start 2024-04-25T11:19:04.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaf00b0b60 0x7fdaf00b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf014b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf015fdc0 0x7fdaf0156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf016da20 con 0x7fdaf00b0b60 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf016d8a0 con 0x7fdaf015fdc0 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf016dba0 con 0x7fdaf00b5730 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaeffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf014b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaeffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf014b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33736/0 (socket says 172.21.15.134:33736) 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaeffff640 1 -- 172.21.15.134:0/774777595 learned_addr learned my addr 172.21.15.134:0/774777595 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf53c3640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf015fdc0 0x7fdaf0156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf4bc2640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaf00b0b60 0x7fdaf00b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf53c3640 1 -- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 msgr2=0x7fdaf014b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf53c3640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf014b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf53c3640 1 -- 172.21.15.134:0/774777595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaf00b0b60 msgr2=0x7fdaf00b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf53c3640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaf00b0b60 0x7fdaf00b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf53c3640 1 -- 172.21.15.134:0/774777595 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf014b550 con 0x7fdaf015fdc0 2024-04-25T11:19:04.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaeffff640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf014b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf53c3640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf015fdc0 0x7fdaf0156500 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdae800bdf0 tx=0x7fdae8009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaed7fa640 1 -- 172.21.15.134:0/774777595 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8019070 con 0x7fdaf015fdc0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaed7fa640 1 -- 172.21.15.134:0/774777595 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae8014440 con 0x7fdaf015fdc0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf014ca30 con 0x7fdaf015fdc0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaed7fa640 1 -- 172.21.15.134:0/774777595 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8014830 con 0x7fdaf015fdc0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf014cff0 con 0x7fdaf015fdc0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaed7fa640 1 -- 172.21.15.134:0/774777595 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdae801c070 con 0x7fdaf015fdc0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaed7fa640 1 -- 172.21.15.134:0/774777595 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fdae801ea80 con 0x7fdaf015fdc0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.079+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf00b3c80 con 0x7fdaf015fdc0 2024-04-25T11:19:04.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.083+0000 7fdaed7fa640 1 -- 172.21.15.134:0/774777595 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdae801e4a0 con 0x7fdaf015fdc0 2024-04-25T11:19:04.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.147+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "245cbf8d-6718-4404-be12-ac042327c299", "id": 7} v 0) -- 0x7fdaf0050750 con 0x7fdaf015fdc0 2024-04-25T11:19:04.153 INFO:tasks.ceph.mgr.y.smithi106.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T11:19:04.153 INFO:tasks.ceph.mgr.y.smithi106.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T11:19:04.153 INFO:tasks.ceph.mgr.y.smithi106.stderr: from numpy import show_config as show_numpy_config 2024-04-25T11:19:04.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaed7fa640 1 -- 172.21.15.134:0/774777595 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "245cbf8d-6718-4404-be12-ac042327c299", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fdae80149d0 con 0x7fdaf015fdc0 2024-04-25T11:19:04.158 INFO:teuthology.orchestra.run.smithi134.stdout:7 2024-04-25T11:19:04.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf015fdc0 msgr2=0x7fdaf0156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf015fdc0 0x7fdaf0156500 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdae800bdf0 tx=0x7fdae8009ef0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 shutdown_connections 2024-04-25T11:19:04.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf015fdc0 0x7fdaf0156500 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf00b5730 0x7fdaf014b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 --2- 172.21.15.134:0/774777595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaf00b0b60 0x7fdaf00b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 >> 172.21.15.134:0/774777595 conn(0x7fdaf009ba30 msgr2=0x7fdaf01616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:04.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 shutdown_connections 2024-04-25T11:19:04.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:04.159+0000 7fdaf6e26640 1 -- 172.21.15.134:0/774777595 wait complete. 2024-04-25T11:19:04.176 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T11:19:04.176 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T11:19:04.179 INFO:tasks.ceph.osd.0:Started 2024-04-25T11:19:04.179 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T11:19:04.179 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T11:19:04.184 INFO:tasks.ceph.osd.1:Started 2024-04-25T11:19:04.184 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T11:19:04.184 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T11:19:04.186 INFO:tasks.ceph.osd.2:Started 2024-04-25T11:19:04.186 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T11:19:04.187 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T11:19:04.189 INFO:tasks.ceph.mgr.x.smithi134.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T11:19:04.189 INFO:tasks.ceph.mgr.x.smithi134.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T11:19:04.190 INFO:tasks.ceph.mgr.x.smithi134.stderr: from numpy import show_config as show_numpy_config 2024-04-25T11:19:04.190 INFO:tasks.ceph.osd.3:Started 2024-04-25T11:19:04.190 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T11:19:04.190 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T11:19:04.192 INFO:tasks.ceph.osd.4:Started 2024-04-25T11:19:04.192 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T11:19:04.193 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T11:19:04.195 INFO:tasks.ceph.osd.5:Started 2024-04-25T11:19:04.195 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T11:19:04.195 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T11:19:04.197 INFO:tasks.ceph.osd.6:Started 2024-04-25T11:19:04.197 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T11:19:04.197 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T11:19:04.200 INFO:tasks.ceph.osd.7:Started 2024-04-25T11:19:04.200 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:04.214 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:04.218+0000 7efcd4851180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:19:04.248 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:04.247+0000 7f0b96ae1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:19:04.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.350+0000 7f016d925640 1 Processor -- start 2024-04-25T11:19:04.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.350+0000 7f016d925640 1 -- start start 2024-04-25T11:19:04.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f016d925640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01680d1980 0x7f01680d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f016d925640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01680d22d0 0x7f01680ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f016d925640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01680d0fa0 con 0x7f016806e090 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f016d925640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f016809cfc0 con 0x7f01680d1980 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f016d925640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f016806da60 con 0x7f01680d22d0 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f01667fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01680d1980 0x7f01680d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f0166ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f016806e090 0x7f016806e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39644/0 (socket says 172.21.15.106:39644) 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f0166ffd640 1 -- 172.21.15.106:0/2727818125 learned_addr learned my addr 172.21.15.106:0/2727818125 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f01677fe640 1 --2- 172.21.15.106:0/2727818125 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01680d22d0 0x7f01680ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f01667fc640 1 -- 172.21.15.106:0/2727818125 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01680d22d0 msgr2=0x7f01680ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f01667fc640 1 --2- 172.21.15.106:0/2727818125 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01680d22d0 0x7f01680ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f01667fc640 1 -- 172.21.15.106:0/2727818125 >> v1:172.21.15.106:6789/0 conn(0x7f016806e090 legacy=0x7f016806e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f01667fc640 1 -- 172.21.15.106:0/2727818125 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016809cae0 con 0x7f01680d1980 2024-04-25T11:19:04.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f0165ffb640 1 -- 172.21.15.106:0/2727818125 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717032301 0 0) 0x7f01680d0fa0 con 0x7f016806e090 2024-04-25T11:19:04.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f01667fc640 1 --2- 172.21.15.106:0/2727818125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d1980 0x7f01680d1d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f01580097c0 tx=0x7f015801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2380f56540da45e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:04.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f0165ffb640 1 -- 172.21.15.106:0/2727818125 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01580b0070 con 0x7f01680d1980 2024-04-25T11:19:04.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f0165ffb640 1 -- 172.21.15.106:0/2727818125 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01580ab600 con 0x7f01680d1980 2024-04-25T11:19:04.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.354+0000 7f0165ffb640 1 -- 172.21.15.106:0/2727818125 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01580ab9d0 con 0x7f01680d1980 2024-04-25T11:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- 172.21.15.106:0/2727818125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d1980 msgr2=0x7f01680d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 --2- 172.21.15.106:0/2727818125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d1980 0x7f01680d1d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f01580097c0 tx=0x7f015801c4a0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- 172.21.15.106:0/2727818125 shutdown_connections 2024-04-25T11:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 --2- 172.21.15.106:0/2727818125 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01680d22d0 0x7f01680ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 --2- 172.21.15.106:0/2727818125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d1980 0x7f01680d1d90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- 172.21.15.106:0/2727818125 >> 172.21.15.106:0/2727818125 conn(0x7f01680d0040 msgr2=0x7f01680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- 172.21.15.106:0/2727818125 shutdown_connections 2024-04-25T11:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- 172.21.15.106:0/2727818125 wait complete. 2024-04-25T11:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 Processor -- start 2024-04-25T11:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- start start 2024-04-25T11:19:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f016806e090 0x7f016815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01680d1980 0x7f016815eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d22d0 0x7f0168163890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01680d26e0 con 0x7f01680d22d0 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f016806ebb0 con 0x7f01680d1980 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f016d925640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01680d2860 con 0x7f016806e090 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f01677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d22d0 0x7f0168163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f01677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d22d0 0x7f0168163890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35666/0 (socket says 172.21.15.106:35666) 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f01677fe640 1 -- 172.21.15.106:0/1495603165 learned_addr learned my addr 172.21.15.106:0/1495603165 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f0166ffd640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f016806e090 0x7f016815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f01667fc640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01680d1980 0x7f016815eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f01677fe640 1 -- 172.21.15.106:0/1495603165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f016806e090 msgr2=0x7f016815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.358+0000 7f01677fe640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f016806e090 0x7f016815f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f01677fe640 1 -- 172.21.15.106:0/1495603165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01680d1980 msgr2=0x7f016815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f01677fe640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01680d1980 0x7f016815eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f01677fe640 1 -- 172.21.15.106:0/1495603165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016815f090 con 0x7f01680d22d0 2024-04-25T11:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f01677fe640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d22d0 0x7f0168163890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f016000d550 tx=0x7f016000da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f014ffff640 1 -- 172.21.15.106:0/1495603165 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01600125f0 con 0x7f01680d22d0 2024-04-25T11:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f014ffff640 1 -- 172.21.15.106:0/1495603165 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0160019070 con 0x7f01680d22d0 2024-04-25T11:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016815faf0 con 0x7f01680d22d0 2024-04-25T11:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f014ffff640 1 -- 172.21.15.106:0/1495603165 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01600129d0 con 0x7f01680d22d0 2024-04-25T11:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681600b0 con 0x7f01680d22d0 2024-04-25T11:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f014ffff640 1 -- 172.21.15.106:0/1495603165 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0160012b70 con 0x7f01680d22d0 2024-04-25T11:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01680b90f0 con 0x7f01680d22d0 2024-04-25T11:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.362+0000 7f014ffff640 1 -- 172.21.15.106:0/1495603165 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f016001eb30 con 0x7f01680d22d0 2024-04-25T11:19:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.366+0000 7f014ffff640 1 -- 172.21.15.106:0/1495603165 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f016001e460 con 0x7f01680d22d0 2024-04-25T11:19:04.386 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:04.386+0000 7efcd4851180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:19:04.407 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:04.407+0000 7f0b96ae1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:19:04.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.418+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0168167f60 con 0x7f01680d22d0 2024-04-25T11:19:04.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f014ffff640 1 -- 172.21.15.106:0/1495603165 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f016001e640 con 0x7f01680d22d0 2024-04-25T11:19:04.419 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:04.419 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":10,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:04.152598+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:04.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d22d0 msgr2=0x7f0168163890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d22d0 0x7f0168163890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f016000d550 tx=0x7f016000da10 comp rx=0 tx=0).stop 2024-04-25T11:19:04.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 shutdown_connections 2024-04-25T11:19:04.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01680d22d0 0x7f0168163890 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01680d1980 0x7f016815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 --2- 172.21.15.106:0/1495603165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f016806e090 0x7f016815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 >> 172.21.15.106:0/1495603165 conn(0x7f01680d0040 msgr2=0x7f01680b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:04.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 shutdown_connections 2024-04-25T11:19:04.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:04.422+0000 7f016d925640 1 -- 172.21.15.106:0/1495603165 wait complete. 2024-04-25T11:19:04.437 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T11:19:04.437 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T11:19:04.546 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:04.550+0000 7efcd4851180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:19:04.566 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:04.567+0000 7f0b96ae1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:19:04.610 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:04.610+0000 7efcd4851180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:19:04.629 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:04.631+0000 7f0b96ae1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:19:04.744 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:04.746+0000 7efcd4851180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:19:04.763 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:04.763+0000 7f0b96ae1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:19:04.818 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-25T11:19:04.819+0000 7f9bc5a947c0 -1 Falling back to public interface 2024-04-25T11:19:04.821 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-25T11:19:04.823+0000 7f9f59a0b7c0 -1 Falling back to public interface 2024-04-25T11:19:04.826 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-25T11:19:04.827+0000 7f91142d57c0 -1 Falling back to public interface 2024-04-25T11:19:04.841 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-25T11:19:04.843+0000 7fd9f35177c0 -1 Falling back to public interface 2024-04-25T11:19:04.851 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-25T11:19:04.854+0000 7fbe068e17c0 -1 Falling back to public interface 2024-04-25T11:19:04.864 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-25T11:19:04.866+0000 7fa8908fb7c0 -1 Falling back to public interface 2024-04-25T11:19:04.864 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-25T11:19:04.866+0000 7f85d65277c0 -1 Falling back to public interface 2024-04-25T11:19:04.875 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-25T11:19:04.878+0000 7f34f806b7c0 -1 Falling back to public interface 2024-04-25T11:19:05.243 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:05.371 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:05.374+0000 7efcd4851180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:19:05.377 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:05.379+0000 7f0b96ae1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:19:05.380 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e60c89640 1 Processor -- start 2024-04-25T11:19:05.380 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e60c89640 1 -- start start 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e60c89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c15dd60 0x7f7e5c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e60c89640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7e5c0b2890 0x7f7e5c0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e60c89640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c15e170 con 0x7f7e5c0b2ca0 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e60c89640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5c06d3e0 con 0x7f7e5c0b2890 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e60c89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c06ce20 con 0x7f7e5c15dd60 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e5a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c15dd60 0x7f7e5c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e5ad9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7e5c0b2ca0 0x7f7e5c0ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:42082/0 (socket says 172.21.15.106:42082) 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e59d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7e5c0b2890 0x7f7e5c0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.382+0000 7f7e5ad9d640 1 -- 172.21.15.106:0/955069023 learned_addr learned my addr 172.21.15.106:0/955069023 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5959a640 1 -- 172.21.15.106:0/955069023 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138482931 0 0) 0x7f7e540014a0 con 0x7f7e5c0b2ca0 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5959a640 1 -- 172.21.15.106:0/955069023 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e48003530 con 0x7f7e5c0b2ca0 2024-04-25T11:19:05.383 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5a59c640 1 -- 172.21.15.106:0/955069023 >> v2:172.21.15.134:3300/0 conn(0x7f7e5c0b2890 msgr2=0x7f7e5c0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:05.384 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5a59c640 1 --2- 172.21.15.106:0/955069023 >> v2:172.21.15.134:3300/0 conn(0x7f7e5c0b2890 0x7f7e5c0b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.384 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5a59c640 1 -- 172.21.15.106:0/955069023 >> v1:172.21.15.134:6789/0 conn(0x7f7e5c0b2ca0 legacy=0x7f7e5c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.384 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5a59c640 1 -- 172.21.15.106:0/955069023 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c06d100 con 0x7f7e5c15dd60 2024-04-25T11:19:05.384 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e59d9b640 1 --2- 172.21.15.106:0/955069023 >> v2:172.21.15.134:3300/0 conn(0x7f7e5c0b2890 0x7f7e5c0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:05.384 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5a59c640 1 --2- 172.21.15.106:0/955069023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c15dd60 0x7f7e5c06c5f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0097c0 tx=0x7f7e4c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85c6eecb77e73df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:05.385 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5959a640 1 -- 172.21.15.106:0/955069023 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0b0070 con 0x7f7e5c15dd60 2024-04-25T11:19:05.385 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5959a640 1 -- 172.21.15.106:0/955069023 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c0ab600 con 0x7f7e5c15dd60 2024-04-25T11:19:05.385 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e5959a640 1 -- 172.21.15.106:0/955069023 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0ab9b0 con 0x7f7e5c15dd60 2024-04-25T11:19:05.386 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e60c89640 1 -- 172.21.15.106:0/955069023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c15dd60 msgr2=0x7f7e5c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.386 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e60c89640 1 --2- 172.21.15.106:0/955069023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c15dd60 0x7f7e5c06c5f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0097c0 tx=0x7f7e4c01c3e0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e60c89640 1 -- 172.21.15.106:0/955069023 shutdown_connections 2024-04-25T11:19:05.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e60c89640 1 --2- 172.21.15.106:0/955069023 >> v2:172.21.15.134:3300/0 conn(0x7f7e5c0b2890 0x7f7e5c0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e60c89640 1 --2- 172.21.15.106:0/955069023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c15dd60 0x7f7e5c06c5f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e60c89640 1 -- 172.21.15.106:0/955069023 >> 172.21.15.106:0/955069023 conn(0x7f7e5c0d2590 msgr2=0x7f7e5c0d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:05.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e60c89640 1 -- 172.21.15.106:0/955069023 shutdown_connections 2024-04-25T11:19:05.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.386+0000 7f7e60c89640 1 -- 172.21.15.106:0/955069023 wait complete. 2024-04-25T11:19:05.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 Processor -- start 2024-04-25T11:19:05.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 -- start start 2024-04-25T11:19:05.391 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c0b2890 0x7f7e5c0d1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.391 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c0b2ca0 0x7f7e5c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.391 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c15dd60 0x7f7e5c153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.391 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c16b420 con 0x7f7e5c15dd60 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c06cb30 con 0x7f7e5c0b2ca0 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c16b5a0 con 0x7f7e5c0b2890 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c15dd60 0x7f7e5c153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c15dd60 0x7f7e5c153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35680/0 (socket says 172.21.15.106:35680) 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 -- 172.21.15.106:0/614947791 learned_addr learned my addr 172.21.15.106:0/614947791 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e59d9b640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c0b2ca0 0x7f7e5c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5a59c640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c0b2890 0x7f7e5c0d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 -- 172.21.15.106:0/614947791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c0b2890 msgr2=0x7f7e5c0d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c0b2890 0x7f7e5c0d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 -- 172.21.15.106:0/614947791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c0b2ca0 msgr2=0x7f7e5c158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c0b2ca0 0x7f7e5c158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 -- 172.21.15.106:0/614947791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0d05f0 con 0x7f7e5c15dd60 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e5ad9d640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c15dd60 0x7f7e5c153060 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400caa0 tx=0x7f7e5400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e52ffd640 1 -- 172.21.15.106:0/614947791 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54017070 con 0x7f7e5c15dd60 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e52ffd640 1 -- 172.21.15.106:0/614947791 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5400ac00 con 0x7f7e5c15dd60 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c0d08d0 con 0x7f7e5c15dd60 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.390+0000 7f7e52ffd640 1 -- 172.21.15.106:0/614947791 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54012400 con 0x7f7e5c15dd60 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.394+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c0d0e90 con 0x7f7e5c15dd60 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.394+0000 7f7e52ffd640 1 -- 172.21.15.106:0/614947791 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7e5401a070 con 0x7f7e5c15dd60 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.394+0000 7f7e52ffd640 1 -- 172.21.15.106:0/614947791 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7e54007740 con 0x7f7e5c15dd60 2024-04-25T11:19:05.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.394+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c050750 con 0x7f7e5c15dd60 2024-04-25T11:19:05.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.394+0000 7f7e52ffd640 1 -- 172.21.15.106:0/614947791 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7e5c15e170 con 0x7f7e5c15dd60 2024-04-25T11:19:05.447 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e5c0b3e60 con 0x7f7e5c15dd60 2024-04-25T11:19:05.448 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e52ffd640 1 -- 172.21.15.106:0/614947791 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7e54007a30 con 0x7f7e5c15dd60 2024-04-25T11:19:05.448 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-25T11:19:05.449 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":10,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:04.152598+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:05.449 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:05.450+0000 7efcd4851180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:19:05.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c15dd60 msgr2=0x7f7e5c153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c15dd60 0x7f7e5c153060 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400caa0 tx=0x7f7e5400cf60 comp rx=0 tx=0).stop 2024-04-25T11:19:05.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 shutdown_connections 2024-04-25T11:19:05.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c15dd60 0x7f7e5c153060 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c0b2ca0 0x7f7e5c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 --2- 172.21.15.106:0/614947791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e5c0b2890 0x7f7e5c0d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 >> 172.21.15.106:0/614947791 conn(0x7f7e5c0d2590 msgr2=0x7f7e5c1548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:05.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 shutdown_connections 2024-04-25T11:19:05.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:05.450+0000 7f7e60c89640 1 -- 172.21.15.106:0/614947791 wait complete. 2024-04-25T11:19:05.454 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:05.455+0000 7f0b96ae1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:19:05.465 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T11:19:05.632 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:05.634+0000 7efcd4851180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:19:05.635 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:05.635+0000 7f0b96ae1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:19:05.705 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:05.706+0000 7efcd4851180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:19:05.708 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:05.707+0000 7f0b96ae1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:19:05.993 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:05.993+0000 7efcd4851180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:19:05.996 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:05.995+0000 7f0b96ae1180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:19:06.055 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:06.057+0000 7efcd4851180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:19:06.058 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T11:19:06.059+0000 7f0b96ae1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:19:07.103 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:07.105+0000 7efc7b300640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T11:19:07.103 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T11:19:07.105+0000 7efc7b300640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T11:19:08.604 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-25T11:19:08.605+0000 7fa8908fb7c0 -1 osd.0 0 log_to_monitors true 2024-04-25T11:19:08.604 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-25T11:19:08.603+0000 7f9bc5a947c0 -1 osd.5 0 log_to_monitors true 2024-04-25T11:19:08.650 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-25T11:19:08.651+0000 7f91142d57c0 -1 osd.7 0 log_to_monitors true 2024-04-25T11:19:08.679 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-25T11:19:08.681+0000 7f34f806b7c0 -1 osd.3 0 log_to_monitors true 2024-04-25T11:19:08.688 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-25T11:19:08.687+0000 7fd9f35177c0 -1 osd.6 0 log_to_monitors true 2024-04-25T11:19:08.709 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-25T11:19:08.711+0000 7f9f59a0b7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T11:19:08.754 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-25T11:19:08.753+0000 7f85d65277c0 -1 osd.2 0 log_to_monitors true 2024-04-25T11:19:08.759 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-25T11:19:08.761+0000 7fbe068e17c0 -1 osd.1 0 log_to_monitors true 2024-04-25T11:19:12.245 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-25T11:19:12.247+0000 7f9f559b2640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T11:19:12.264 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-25T11:19:12.263+0000 7f9bc1a3b640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T11:19:12.272 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:12.273 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-25T11:19:12.275+0000 7f9f4d7b3640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:12.282 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-25T11:19:12.283+0000 7fd9ef4be640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T11:19:12.289 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-25T11:19:12.291+0000 7f9bb983c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:12.307 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-25T11:19:12.309+0000 7f34f4012640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T11:19:12.313 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-25T11:19:12.315+0000 7fd9e72bf640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:12.323 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-25T11:19:12.323+0000 7f911027c640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T11:19:12.343 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-25T11:19:12.345+0000 7f34ebe13640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:12.361 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-25T11:19:12.363+0000 7f910807d640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:12.378 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-25T11:19:12.381+0000 7fa88c8a2640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T11:19:12.383 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-25T11:19:12.385+0000 7f85d24ce640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T11:19:12.384 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-25T11:19:12.385+0000 7fbe02888640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T11:19:12.407 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-25T11:19:12.409+0000 7fa8846a3640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:12.419 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-25T11:19:12.421+0000 7f85ca2cf640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:12.419 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-25T11:19:12.421+0000 7fbdfa689640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:12.441 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.441+0000 7fe54866a640 1 Processor -- start 2024-04-25T11:19:12.441 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.441+0000 7fe54866a640 1 -- start start 2024-04-25T11:19:12.443 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.441+0000 7fe54866a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400cdc00 0x7fe5400ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.443 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.441+0000 7fe54866a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5400cec10 0x7fe5400cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.443 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.441+0000 7fe54866a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe5400d0370 0x7fe5400d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.443 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.441+0000 7fe54866a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe5400b53e0 con 0x7fe5400cec10 2024-04-25T11:19:12.443 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.441+0000 7fe54866a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe5400b5f60 con 0x7fe5400d0370 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.441+0000 7fe54866a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400cf560 con 0x7fe5400cdc00 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5400cec10 0x7fe5400cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5400cec10 0x7fe5400cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47150/0 (socket says 172.21.15.106:47150) 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 -- 172.21.15.106:0/3797180771 learned_addr learned my addr 172.21.15.106:0/3797180771 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe546406640 1 --2- 172.21.15.106:0/3797180771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400cdc00 0x7fe5400ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe546c07640 1 --2- 172.21.15.106:0/3797180771 >> v2:172.21.15.134:3300/0 conn(0x7fe5400d0370 0x7fe5400d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 -- 172.21.15.106:0/3797180771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400cdc00 msgr2=0x7fe5400ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 --2- 172.21.15.106:0/3797180771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400cdc00 0x7fe5400ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 -- 172.21.15.106:0/3797180771 >> v2:172.21.15.134:3300/0 conn(0x7fe5400d0370 msgr2=0x7fe5400d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 --2- 172.21.15.106:0/3797180771 >> v2:172.21.15.134:3300/0 conn(0x7fe5400d0370 0x7fe5400d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.444 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 -- 172.21.15.106:0/3797180771 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5400b5630 con 0x7fe5400cec10 2024-04-25T11:19:12.445 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545c05640 1 --2- 172.21.15.106:0/3797180771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cec10 0x7fe5400cf020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe540073ae0 tx=0x7fe53001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0892386750f31b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:12.445 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545404640 1 -- 172.21.15.106:0/3797180771 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5300b5070 con 0x7fe5400cec10 2024-04-25T11:19:12.445 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545404640 1 -- 172.21.15.106:0/3797180771 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5300a89d0 con 0x7fe5400cec10 2024-04-25T11:19:12.445 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe545404640 1 -- 172.21.15.106:0/3797180771 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5300a8da0 con 0x7fe5400cec10 2024-04-25T11:19:12.446 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 -- 172.21.15.106:0/3797180771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cec10 msgr2=0x7fe5400cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.446 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 --2- 172.21.15.106:0/3797180771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cec10 0x7fe5400cf020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe540073ae0 tx=0x7fe53001ed10 comp rx=0 tx=0).stop 2024-04-25T11:19:12.446 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 -- 172.21.15.106:0/3797180771 shutdown_connections 2024-04-25T11:19:12.446 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 --2- 172.21.15.106:0/3797180771 >> v2:172.21.15.134:3300/0 conn(0x7fe5400d0370 0x7fe5400d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.446 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 --2- 172.21.15.106:0/3797180771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cec10 0x7fe5400cf020 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.446 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 --2- 172.21.15.106:0/3797180771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400cdc00 0x7fe5400ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.447 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 -- 172.21.15.106:0/3797180771 >> 172.21.15.106:0/3797180771 conn(0x7fe5400d2590 msgr2=0x7fe5400d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:12.447 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 -- 172.21.15.106:0/3797180771 shutdown_connections 2024-04-25T11:19:12.447 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.445+0000 7fe54866a640 1 -- 172.21.15.106:0/3797180771 wait complete. 2024-04-25T11:19:12.447 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 Processor -- start 2024-04-25T11:19:12.447 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 -- start start 2024-04-25T11:19:12.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cdc00 0x7fe5400af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.450 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5400cec10 0x7fe5400b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400d0370 0x7fe540148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400cf900 con 0x7fe5400cdc00 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400b5bf0 con 0x7fe5400cec10 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400cfa80 con 0x7fe5400d0370 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cdc00 0x7fe5400af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cdc00 0x7fe5400af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47158/0 (socket says 172.21.15.106:47158) 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 -- 172.21.15.106:0/3944875540 learned_addr learned my addr 172.21.15.106:0/3944875540 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe545c05640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5400cec10 0x7fe5400b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546c07640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400d0370 0x7fe540148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 -- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400d0370 msgr2=0x7fe540148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400d0370 0x7fe540148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 -- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5400cec10 msgr2=0x7fe5400b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5400cec10 0x7fe5400b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.451 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 -- 172.21.15.106:0/3944875540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe540147c80 con 0x7fe5400cdc00 2024-04-25T11:19:12.452 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe545c05640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5400cec10 0x7fe5400b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:12.452 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe546406640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cdc00 0x7fe5400af290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00d800 tx=0x7fe53c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:12.452 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe536ffd640 1 -- 172.21.15.106:0/3944875540 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c018030 con 0x7fe5400cdc00 2024-04-25T11:19:12.452 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe536ffd640 1 -- 172.21.15.106:0/3944875540 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53c0096a0 con 0x7fe5400cdc00 2024-04-25T11:19:12.452 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe540147f60 con 0x7fe5400cdc00 2024-04-25T11:19:12.452 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe536ffd640 1 -- 172.21.15.106:0/3944875540 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c009870 con 0x7fe5400cdc00 2024-04-25T11:19:12.452 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.449+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe540148520 con 0x7fe5400cdc00 2024-04-25T11:19:12.452 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.453+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe540050750 con 0x7fe5400cdc00 2024-04-25T11:19:12.453 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.453+0000 7fe536ffd640 1 -- 172.21.15.106:0/3944875540 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) ==== 93151+0+0 (secure 0 0 0) 0x7fe53c01b070 con 0x7fe5400cdc00 2024-04-25T11:19:12.453 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.453+0000 7fe536ffd640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe52406f370 0x7fe524071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.453 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.453+0000 7fe536ffd640 1 -- 172.21.15.106:0/3944875540 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fe53c08e6d0 con 0x7fe5400cdc00 2024-04-25T11:19:12.453 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.453+0000 7fe545c05640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe52406f370 0x7fe524071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.453 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.453+0000 7fe545c05640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe52406f370 0x7fe524071880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe5300b2000 tx=0x7fe530008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:12.456 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.457+0000 7fe536ffd640 1 -- 172.21.15.106:0/3944875540 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe53c05c4c0 con 0x7fe5400cdc00 2024-04-25T11:19:12.605 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.605+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5400b80c0 con 0x7fe5400cdc00 2024-04-25T11:19:12.606 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.605+0000 7fe536ffd640 1 -- 172.21.15.106:0/3944875540 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fe53c05bdc0 con 0x7fe5400cdc00 2024-04-25T11:19:12.606 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-25T11:19:12.606 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":12,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:10.104571+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:12.608 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe52406f370 msgr2=0x7fe524071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.608 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe52406f370 0x7fe524071880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe5300b2000 tx=0x7fe530008040 comp rx=0 tx=0).stop 2024-04-25T11:19:12.608 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cdc00 msgr2=0x7fe5400af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cdc00 0x7fe5400af290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00d800 tx=0x7fe53c00dcc0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 shutdown_connections 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5400d0370 0x7fe540148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5400cec10 0x7fe5400b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe52406f370 0x7fe524071880 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 --2- 172.21.15.106:0/3944875540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5400cdc00 0x7fe5400af290 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 >> 172.21.15.106:0/3944875540 conn(0x7fe5400d2590 msgr2=0x7fe5400b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 shutdown_connections 2024-04-25T11:19:12.609 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:12.609+0000 7fe54866a640 1 -- 172.21.15.106:0/3944875540 wait complete. 2024-04-25T11:19:12.624 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T11:19:19.431 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:19.582 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb38b7d6640 1 Processor -- start 2024-04-25T11:19:19.583 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb38b7d6640 1 -- start start 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb38b7d6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3840d1f40 0x7fb38415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb38b7d6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb38415d900 0x7fb3840b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb38b7d6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3840b45e0 con 0x7fb3840d0e00 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb38b7d6640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb3840b48c0 con 0x7fb3840d1f40 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb38b7d6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb3840b4ba0 con 0x7fb38415d900 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb388d71640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3840d1f40 0x7fb38415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb388d71640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3840d1f40 0x7fb38415d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47872/0 (socket says 172.21.15.106:47872) 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb388d71640 1 -- 172.21.15.106:0/863472774 learned_addr learned my addr 172.21.15.106:0/863472774 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb389d73640 1 --2- 172.21.15.106:0/863472774 >> v2:172.21.15.134:3300/0 conn(0x7fb38415d900 0x7fb3840b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.585 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb388d71640 1 -- 172.21.15.106:0/863472774 >> v2:172.21.15.134:3300/0 conn(0x7fb38415d900 msgr2=0x7fb3840b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:19.586 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb388d71640 1 --2- 172.21.15.106:0/863472774 >> v2:172.21.15.134:3300/0 conn(0x7fb38415d900 0x7fb3840b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.586 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb388d71640 1 -- 172.21.15.106:0/863472774 >> v1:172.21.15.134:6789/0 conn(0x7fb3840d0e00 legacy=0x7fb3840d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:19.586 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.585+0000 7fb388d71640 1 -- 172.21.15.106:0/863472774 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3840b4e80 con 0x7fb3840d1f40 2024-04-25T11:19:19.587 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb388d71640 1 --2- 172.21.15.106:0/863472774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d1f40 0x7fb38415d3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3840b0c60 tx=0x7fb37401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b14ad08c8bb853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.587 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb37bfff640 1 -- 172.21.15.106:0/863472774 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3740b2070 con 0x7fb3840d1f40 2024-04-25T11:19:19.587 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb37bfff640 1 -- 172.21.15.106:0/863472774 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3740a9070 con 0x7fb3840d1f40 2024-04-25T11:19:19.587 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb37bfff640 1 -- 172.21.15.106:0/863472774 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3740ad880 con 0x7fb3840d1f40 2024-04-25T11:19:19.588 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 -- 172.21.15.106:0/863472774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d1f40 msgr2=0x7fb38415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.588 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/863472774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d1f40 0x7fb38415d3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3840b0c60 tx=0x7fb37401c450 comp rx=0 tx=0).stop 2024-04-25T11:19:19.588 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 -- 172.21.15.106:0/863472774 shutdown_connections 2024-04-25T11:19:19.588 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/863472774 >> v2:172.21.15.134:3300/0 conn(0x7fb38415d900 0x7fb3840b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.588 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/863472774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d1f40 0x7fb38415d3c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.588 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 -- 172.21.15.106:0/863472774 >> 172.21.15.106:0/863472774 conn(0x7fb38406c9f0 msgr2=0x7fb38406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:19.588 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 -- 172.21.15.106:0/863472774 shutdown_connections 2024-04-25T11:19:19.588 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 -- 172.21.15.106:0/863472774 wait complete. 2024-04-25T11:19:19.590 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 Processor -- start 2024-04-25T11:19:19.590 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.589+0000 7fb38b7d6640 1 -- start start 2024-04-25T11:19:19.592 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d0e00 0x7fb3840b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb38415d900 0x7fb384150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3841506b0 0x7fb384159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb384169a00 con 0x7fb3840d0e00 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb384169880 con 0x7fb3841506b0 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb384169b80 con 0x7fb38415d900 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d0e00 0x7fb3840b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d0e00 0x7fb3840b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47874/0 (socket says 172.21.15.106:47874) 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 -- 172.21.15.106:0/1988271402 learned_addr learned my addr 172.21.15.106:0/1988271402 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb388d71640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb38415d900 0x7fb384150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389d73640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3841506b0 0x7fb384159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 -- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb38415d900 msgr2=0x7fb384150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb38415d900 0x7fb384150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.593 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 -- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3841506b0 msgr2=0x7fb384159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3841506b0 0x7fb384159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 -- 172.21.15.106:0/1988271402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3840af280 con 0x7fb3840d0e00 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389d73640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3841506b0 0x7fb384159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb389572640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d0e00 0x7fb3840b02b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b410 tx=0x7fb38000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb379ffb640 1 -- 172.21.15.106:0/1988271402 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380015030 con 0x7fb3840d0e00 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb379ffb640 1 -- 172.21.15.106:0/1988271402 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb380004020 con 0x7fb3840d0e00 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb379ffb640 1 -- 172.21.15.106:0/1988271402 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380004450 con 0x7fb3840d0e00 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb384151140 con 0x7fb3840d0e00 2024-04-25T11:19:19.594 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3840af7d0 con 0x7fb3840d0e00 2024-04-25T11:19:19.595 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.593+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb384050750 con 0x7fb3840d0e00 2024-04-25T11:19:19.595 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.597+0000 7fb379ffb640 1 -- 172.21.15.106:0/1988271402 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb380018070 con 0x7fb3840d0e00 2024-04-25T11:19:19.595 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.597+0000 7fb379ffb640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb36806f400 0x7fb368071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.595 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.597+0000 7fb388d71640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb36806f400 0x7fb368071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.595 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.597+0000 7fb379ffb640 1 -- 172.21.15.106:0/1988271402 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb38008ebf0 con 0x7fb3840d0e00 2024-04-25T11:19:19.595 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.597+0000 7fb388d71640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb36806f400 0x7fb368071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb3740af000 tx=0x7fb374008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.600 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.601+0000 7fb379ffb640 1 -- 172.21.15.106:0/1988271402 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb38005bd90 con 0x7fb3840d0e00 2024-04-25T11:19:19.759 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.761+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb38415caf0 con 0x7fb3840d0e00 2024-04-25T11:19:19.761 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.761+0000 7fb379ffb640 1 -- 172.21.15.106:0/1988271402 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7fb38005b690 con 0x7fb3840d0e00 2024-04-25T11:19:19.761 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-25T11:19:19.761 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":16,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:16.328802+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:19.763 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb36806f400 msgr2=0x7fb368071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.763 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb36806f400 0x7fb368071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb3740af000 tx=0x7fb374008040 comp rx=0 tx=0).stop 2024-04-25T11:19:19.763 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d0e00 msgr2=0x7fb3840b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d0e00 0x7fb3840b02b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b410 tx=0x7fb38000b8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 shutdown_connections 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3841506b0 0x7fb384159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb38415d900 0x7fb384150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb36806f400 0x7fb368071910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 --2- 172.21.15.106:0/1988271402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3840d0e00 0x7fb3840b02b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 >> 172.21.15.106:0/1988271402 conn(0x7fb38406c9f0 msgr2=0x7fb38406f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 shutdown_connections 2024-04-25T11:19:19.764 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T11:19:19.765+0000 7fb38b7d6640 1 -- 172.21.15.106:0/1988271402 wait complete. 2024-04-25T11:19:19.780 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T11:19:19.780 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T11:19:19.781 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:19:19.781 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:19:19.783 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:19:19.783 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:19:19.783 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:19:19.785 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:19:19.785 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:19:19.785 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:19:19.788 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:19:19.788 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:19:19.788 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:19:19.791 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:19:19.791 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T11:19:19.791 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:19.895 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-25T11:19:19.896 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-25T11:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992c40a640 1 Processor -- start 2024-04-25T11:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992c40a640 1 -- start start 2024-04-25T11:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992c40a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99240ba970 0x7f99240d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992c40a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99240d0530 con 0x7f99240b5120 2024-04-25T11:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992c40a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99240b5c40 con 0x7f99240d2160 2024-04-25T11:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992c40a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f99240d1100 con 0x7f99240ba970 2024-04-25T11:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992a9a7640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f99240b5120 0x7f99240b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59476/0 (socket says 172.21.15.106:59476) 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992a9a7640 1 -- 172.21.15.106:0/3810316367 learned_addr learned my addr 172.21.15.106:0/3810316367 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f992a1a6640 1 --2- 172.21.15.106:0/3810316367 >> v2:172.21.15.134:3300/0 conn(0x7f99240ba970 0x7f99240d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 54798273 0 0) 0x7f99240d0530 con 0x7f99240b5120 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9910003530 con 0x7f99240b5120 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2629990526 0 0) 0x7f9910003530 con 0x7f99240b5120 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 >> v2:172.21.15.134:3300/0 conn(0x7f99240ba970 msgr2=0x7f99240d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 --2- 172.21.15.106:0/3810316367 >> v2:172.21.15.134:3300/0 conn(0x7f99240ba970 0x7f99240d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 >> v1:172.21.15.134:6789/0 conn(0x7f99240d2160 legacy=0x7f99240d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99240d07d0 con 0x7f99240b5120 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f991c001a00 con 0x7f99240b5120 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 >> v1:172.21.15.106:6789/0 conn(0x7f99240b5120 legacy=0x7f99240b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99100039c0 0x7f9910003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9910004810 0x7f9910004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99100051d0 0x7f991000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99240d1100 con 0x7f99100039c0 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f991000c170 con 0x7f9910004810 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.933+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f991000c370 con 0x7f99100051d0 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992a1a6640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99100039c0 0x7f9910003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f99299a5640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9910004810 0x7f9910004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992a1a6640 1 -- 172.21.15.106:0/3810316367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99100051d0 msgr2=0x7f991000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992a1a6640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99100051d0 0x7f991000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992a1a6640 1 -- 172.21.15.106:0/3810316367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9910004810 msgr2=0x7f9910004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992a1a6640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9910004810 0x7f9910004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992a1a6640 1 -- 172.21.15.106:0/3810316367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9910003530 con 0x7f99100039c0 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992a9a7640 1 -- 172.21.15.106:0/3810316367 reap_dead start 2024-04-25T11:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f99299a5640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9910004810 0x7f9910004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:19.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992a1a6640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99100039c0 0x7f9910003e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99240bb890 tx=0x7f992001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01d7edce07432aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99200ad070 con 0x7f99100039c0 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99200059a0 con 0x7f99100039c0 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f99291a4640 1 -- 172.21.15.106:0/3810316367 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920009690 con 0x7f99100039c0 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992c40a640 1 -- 172.21.15.106:0/3810316367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99100039c0 msgr2=0x7f9910003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992c40a640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99100039c0 0x7f9910003e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99240bb890 tx=0x7f992001c930 comp rx=0 tx=0).stop 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992c40a640 1 -- 172.21.15.106:0/3810316367 shutdown_connections 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992c40a640 1 --2- 172.21.15.106:0/3810316367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99100039c0 0x7f9910003e30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992c40a640 1 -- 172.21.15.106:0/3810316367 >> 172.21.15.106:0/3810316367 conn(0x7f99240d57e0 msgr2=0x7f99240d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992c40a640 1 -- 172.21.15.106:0/3810316367 shutdown_connections 2024-04-25T11:19:19.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.937+0000 7f992c40a640 1 -- 172.21.15.106:0/3810316367 wait complete. 2024-04-25T11:19:19.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 Processor -- start 2024-04-25T11:19:19.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 -- start start 2024-04-25T11:19:19.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99100039c0 0x7f99241fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99240a2d40 0x7f9924161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9924162260 0x7f99240a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99240d2980 con 0x7f9924162260 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99241605b0 con 0x7f99240a2d40 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f99240a1bf0 con 0x7f99100039c0 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9924162260 0x7f99240a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9924162260 0x7f99240a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47932/0 (socket says 172.21.15.106:47932) 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 -- 172.21.15.106:0/116440404 learned_addr learned my addr 172.21.15.106:0/116440404 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 -- 172.21.15.106:0/116440404 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99100039c0 msgr2=0x7f99241fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99100039c0 0x7f99241fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 -- 172.21.15.106:0/116440404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99240a2d40 msgr2=0x7f9924161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99240a2d40 0x7f9924161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 -- 172.21.15.106:0/116440404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9924162670 con 0x7f9924162260 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992a9a7640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9924162260 0x7f99240a08b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f991c001570 tx=0x7f991c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f991affd640 1 -- 172.21.15.106:0/116440404 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c00a5b0 con 0x7f9924162260 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f991affd640 1 -- 172.21.15.106:0/116440404 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991c00a750 con 0x7f9924162260 2024-04-25T11:19:19.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992409f770 con 0x7f9924162260 2024-04-25T11:19:19.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f991affd640 1 -- 172.21.15.106:0/116440404 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c00ab20 con 0x7f9924162260 2024-04-25T11:19:19.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.941+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992409fd30 con 0x7f9924162260 2024-04-25T11:19:19.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.945+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9924050750 con 0x7f9924162260 2024-04-25T11:19:19.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.945+0000 7f991affd640 1 -- 172.21.15.106:0/116440404 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f991c00acc0 con 0x7f9924162260 2024-04-25T11:19:19.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.945+0000 7f991affd640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f990806f400 0x7f9908071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.945+0000 7f992a1a6640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f990806f400 0x7f9908071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.945+0000 7f991affd640 1 -- 172.21.15.106:0/116440404 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f991c08e910 con 0x7f9924162260 2024-04-25T11:19:19.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.945+0000 7f992a1a6640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f990806f400 0x7f9908071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99200ac000 tx=0x7f9920004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:19.949+0000 7f991affd640 1 -- 172.21.15.106:0/116440404 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9924162670 con 0x7f9924162260 2024-04-25T11:19:19.960 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-04-25T11:19:19.973 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-04-25T11:19:20.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.101+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99240d2ed0 con 0x7f9924162260 2024-04-25T11:19:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.105+0000 7f991affd640 1 -- 172.21.15.106:0/116440404 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f991c05bb50 con 0x7f9924162260 2024-04-25T11:19:20.104 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:20.104 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:16.328802+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:20.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f990806f400 msgr2=0x7f9908071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f990806f400 0x7f9908071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99200ac000 tx=0x7f9920004740 comp rx=0 tx=0).stop 2024-04-25T11:19:20.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9924162260 msgr2=0x7f99240a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9924162260 0x7f99240a08b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f991c001570 tx=0x7f991c00cf00 comp rx=0 tx=0).stop 2024-04-25T11:19:20.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 shutdown_connections 2024-04-25T11:19:20.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f990806f400 0x7f9908071910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9924162260 0x7f99240a08b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99240a2d40 0x7f9924161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 --2- 172.21.15.106:0/116440404 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99100039c0 0x7f99241fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 >> 172.21.15.106:0/116440404 conn(0x7f99240d57e0 msgr2=0x7f99240b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:20.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 shutdown_connections 2024-04-25T11:19:20.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.109+0000 7f992c40a640 1 -- 172.21.15.106:0/116440404 wait complete. 2024-04-25T11:19:20.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:20.125 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:20.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.273+0000 7f754267e640 1 Processor -- start 2024-04-25T11:19:20.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.273+0000 7f754267e640 1 -- start start 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f754267e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f753c0d1e50 0x7f753c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f754267e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f753c15d230 0x7f753c15d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f754267e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f753c0b68e0 con 0x7f753c0d0cf0 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f754267e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f753c161210 con 0x7f753c15d230 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f754267e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f753c0b6290 con 0x7f753c0d1e50 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f7540c1b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f753c15d230 0x7f753c15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f7540c1b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f753c15d230 0x7f753c15d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47942/0 (socket says 172.21.15.106:47942) 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f7540c1b640 1 -- 172.21.15.106:0/4217643262 learned_addr learned my addr 172.21.15.106:0/4217643262 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f753b7fe640 1 --2- 172.21.15.106:0/4217643262 >> v2:172.21.15.134:3300/0 conn(0x7f753c0d1e50 0x7f753c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f7540c1b640 1 -- 172.21.15.106:0/4217643262 >> v2:172.21.15.134:3300/0 conn(0x7f753c0d1e50 msgr2=0x7f753c06ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f7540c1b640 1 --2- 172.21.15.106:0/4217643262 >> v2:172.21.15.134:3300/0 conn(0x7f753c0d1e50 0x7f753c06ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f7540c1b640 1 -- 172.21.15.106:0/4217643262 >> v1:172.21.15.134:6789/0 conn(0x7f753c0d0cf0 legacy=0x7f753c0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f7540c1b640 1 -- 172.21.15.106:0/4217643262 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c0b6570 con 0x7f753c15d230 2024-04-25T11:19:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.277+0000 7f7540c1b640 1 --2- 172.21.15.106:0/4217643262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c15d640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f753c0b1010 tx=0x7f753401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb8588a8c9c3d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f753affd640 1 -- 172.21.15.106:0/4217643262 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75340a5590 con 0x7f753c15d230 2024-04-25T11:19:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f753affd640 1 -- 172.21.15.106:0/4217643262 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75340a5730 con 0x7f753c15d230 2024-04-25T11:19:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f753affd640 1 -- 172.21.15.106:0/4217643262 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75340a5b00 con 0x7f753c15d230 2024-04-25T11:19:20.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 -- 172.21.15.106:0/4217643262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 msgr2=0x7f753c15d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 --2- 172.21.15.106:0/4217643262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c15d640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f753c0b1010 tx=0x7f753401c8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 -- 172.21.15.106:0/4217643262 shutdown_connections 2024-04-25T11:19:20.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 --2- 172.21.15.106:0/4217643262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c15d640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 --2- 172.21.15.106:0/4217643262 >> v2:172.21.15.134:3300/0 conn(0x7f753c0d1e50 0x7f753c06ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 -- 172.21.15.106:0/4217643262 >> 172.21.15.106:0/4217643262 conn(0x7f753c06c9f0 msgr2=0x7f753c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:20.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 -- 172.21.15.106:0/4217643262 shutdown_connections 2024-04-25T11:19:20.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 -- 172.21.15.106:0/4217643262 wait complete. 2024-04-25T11:19:20.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 Processor -- start 2024-04-25T11:19:20.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.281+0000 7f754267e640 1 -- start start 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f753c0d0cf0 0x7f753c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c13e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c140e80 0x7f753c141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f753c16a080 con 0x7f753c15d230 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f753c15cdd0 con 0x7f753c140e80 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f753c16a200 con 0x7f753c0d0cf0 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c13e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c13e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47950/0 (socket says 172.21.15.106:47950) 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 -- 172.21.15.106:0/1712695060 learned_addr learned my addr 172.21.15.106:0/1712695060 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f7540c1b640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c140e80 0x7f753c141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753bfff640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f753c0d0cf0 0x7f753c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 -- 172.21.15.106:0/1712695060 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f753c0d0cf0 msgr2=0x7f753c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f753c0d0cf0 0x7f753c158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 -- 172.21.15.106:0/1712695060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c140e80 msgr2=0x7f753c141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c140e80 0x7f753c141290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 -- 172.21.15.106:0/1712695060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c1417d0 con 0x7f753c15d230 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753b7fe640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c13e450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f753000d7d0 tx=0x7f753000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f7538ff9640 1 -- 172.21.15.106:0/1712695060 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753000b6c0 con 0x7f753c15d230 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f7538ff9640 1 -- 172.21.15.106:0/1712695060 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7530019070 con 0x7f753c15d230 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c161b70 con 0x7f753c15d230 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f7538ff9640 1 -- 172.21.15.106:0/1712695060 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753000bac0 con 0x7f753c15d230 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c13f170 con 0x7f753c15d230 2024-04-25T11:19:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c1618e0 con 0x7f753c15d230 2024-04-25T11:19:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f7538ff9640 1 -- 172.21.15.106:0/1712695060 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f753000bc60 con 0x7f753c15d230 2024-04-25T11:19:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f7538ff9640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f752806f430 0x7f7528071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753bfff640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f752806f430 0x7f7528071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f7538ff9640 1 -- 172.21.15.106:0/1712695060 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f753008fdb0 con 0x7f753c15d230 2024-04-25T11:19:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.285+0000 7f753bfff640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f752806f430 0x7f7528071940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f752c007fd0 tx=0x7f752c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:20.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.289+0000 7f7538ff9640 1 -- 172.21.15.106:0/1712695060 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f753c1417d0 con 0x7f753c15d230 2024-04-25T11:19:20.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f753c050750 con 0x7f753c15d230 2024-04-25T11:19:20.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f7538ff9640 1 -- 172.21.15.106:0/1712695060 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f753005d160 con 0x7f753c15d230 2024-04-25T11:19:20.451 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f752806f430 msgr2=0x7f7528071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f752806f430 0x7f7528071940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f752c007fd0 tx=0x7f752c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 msgr2=0x7f753c13e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c13e450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f753000d7d0 tx=0x7f753000dc90 comp rx=0 tx=0).stop 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 shutdown_connections 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c140e80 0x7f753c141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f752806f430 0x7f7528071940 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753c15d230 0x7f753c13e450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 --2- 172.21.15.106:0/1712695060 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f753c0d0cf0 0x7f753c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 >> 172.21.15.106:0/1712695060 conn(0x7f753c06c9f0 msgr2=0x7f753c0b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 shutdown_connections 2024-04-25T11:19:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.453+0000 7f754267e640 1 -- 172.21.15.106:0/1712695060 wait complete. 2024-04-25T11:19:20.473 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:20.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf8488640 1 Processor -- start 2024-04-25T11:19:20.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf8488640 1 -- start start 2024-04-25T11:19:20.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf8488640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 0x7f2bf0073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf8488640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2bf00d0ce0 0x7f2bf00d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf8488640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf00ce050 con 0x7f2bf00ce850 2024-04-25T11:19:20.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf8488640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf015e6b0 con 0x7f2bf00d0ce0 2024-04-25T11:19:20.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf8488640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf015e990 con 0x7f2bf015e0a0 2024-04-25T11:19:20.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5a23640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2bf00ce850 0x7f2bf00cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59492/0 (socket says 172.21.15.106:59492) 2024-04-25T11:19:20.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5a23640 1 -- 172.21.15.106:0/561093861 learned_addr learned my addr 172.21.15.106:0/561093861 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:20.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369480247 0 0) 0x7f2bf00ce050 con 0x7f2bf00ce850 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bd8003530 con 0x7f2bf00ce850 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf6224640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 0x7f2bf0073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf6a25640 1 --2- 172.21.15.106:0/561093861 >> v2:172.21.15.134:3300/0 conn(0x7f2bf00d0ce0 0x7f2bf00d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2101784952 0 0) 0x7f2bd8003530 con 0x7f2bf00ce850 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 msgr2=0x7f2bf0073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 0x7f2bf0073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 >> v2:172.21.15.134:3300/0 conn(0x7f2bf00d0ce0 msgr2=0x7f2bf00d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 --2- 172.21.15.106:0/561093861 >> v2:172.21.15.134:3300/0 conn(0x7f2bf00d0ce0 0x7f2bf00d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf015ec70 con 0x7f2bf00ce850 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f2be4001680 con 0x7f2bf00ce850 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 >> v1:172.21.15.106:6789/0 conn(0x7f2bf00ce850 legacy=0x7f2bf00cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd80039c0 0x7f2bd8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd8004810 0x7f2bd8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 0x7f2bd80051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf015e6b0 con 0x7f2bd8004810 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf015e990 con 0x7f2bd80039c0 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd800bb30 con 0x7f2bf015e0a0 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf5a23640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd8004810 0x7f2bd8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.617+0000 7f2bf6224640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd80039c0 0x7f2bd8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf6a25640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 0x7f2bd80051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5a23640 1 -- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 msgr2=0x7f2bd80051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5a23640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 0x7f2bd80051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5a23640 1 -- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd80039c0 msgr2=0x7f2bd8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5a23640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd80039c0 0x7f2bd8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5a23640 1 -- 172.21.15.106:0/561093861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd8003530 con 0x7f2bd8004810 2024-04-25T11:19:20.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf6a25640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf015e0a0 0x7f2bd80051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5a23640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd8004810 0x7f2bd8004c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c240 tx=0x7f2be401f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7d47c2e3f13cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4007d40 con 0x7f2bd8004810 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be4005ae0 con 0x7f2bd8004810 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf5222640 1 -- 172.21.15.106:0/561093861 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4007070 con 0x7f2bd8004810 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf8488640 1 -- 172.21.15.106:0/561093861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd8004810 msgr2=0x7f2bd8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf8488640 1 --2- 172.21.15.106:0/561093861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd8004810 0x7f2bd8004c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c240 tx=0x7f2be401f410 comp rx=0 tx=0).stop 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf6a25640 1 -- 172.21.15.106:0/561093861 reap_dead start 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf8488640 1 -- 172.21.15.106:0/561093861 shutdown_connections 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf8488640 1 -- 172.21.15.106:0/561093861 >> 172.21.15.106:0/561093861 conn(0x7f2bf006c7c0 msgr2=0x7f2bf006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf8488640 1 -- 172.21.15.106:0/561093861 shutdown_connections 2024-04-25T11:19:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.621+0000 7f2bf8488640 1 -- 172.21.15.106:0/561093861 wait complete. 2024-04-25T11:19:20.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 Processor -- start 2024-04-25T11:19:20.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 -- start start 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd8004810 0x7f2bf0147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf0145030 0x7f2bf01f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf013f780 0x7f2bf013fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf00d0860 con 0x7f2bf0145030 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf00d06e0 con 0x7f2bd8004810 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf015f270 con 0x7f2bf013f780 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf0145030 0x7f2bf01f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf0145030 0x7f2bf01f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47964/0 (socket says 172.21.15.106:47964) 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 -- 172.21.15.106:0/2338455627 learned_addr learned my addr 172.21.15.106:0/2338455627 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf6a25640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf013f780 0x7f2bf013fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf6224640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd8004810 0x7f2bf0147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 -- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf013f780 msgr2=0x7f2bf013fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf013f780 0x7f2bf013fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 -- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd8004810 msgr2=0x7f2bf0147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd8004810 0x7f2bf0147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 -- 172.21.15.106:0/2338455627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf01400d0 con 0x7f2bf0145030 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf6a25640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf013f780 0x7f2bf013fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf5a23640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf0145030 0x7f2bf01f70e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c240 tx=0x7f2bd8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2beeffd640 1 -- 172.21.15.106:0/2338455627 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4005c90 con 0x7f2bf0145030 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2beeffd640 1 -- 172.21.15.106:0/2338455627 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be4005e30 con 0x7f2bf0145030 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf0145710 con 0x7f2bf0145030 2024-04-25T11:19:20.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2beeffd640 1 -- 172.21.15.106:0/2338455627 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40049f0 con 0x7f2bf0145030 2024-04-25T11:19:20.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf0145cd0 con 0x7f2bf0145030 2024-04-25T11:19:20.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.625+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf0143050 con 0x7f2bf0145030 2024-04-25T11:19:20.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.629+0000 7f2beeffd640 1 -- 172.21.15.106:0/2338455627 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2be40ba030 con 0x7f2bf0145030 2024-04-25T11:19:20.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.629+0000 7f2beeffd640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2bdc06f400 0x7f2bdc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:20.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.629+0000 7f2beeffd640 1 -- 172.21.15.106:0/2338455627 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2be412be90 con 0x7f2bf0145030 2024-04-25T11:19:20.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.629+0000 7f2bf6224640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2bdc06f400 0x7f2bdc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:20.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.629+0000 7f2bf6224640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2bdc06f400 0x7f2bdc071910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2be000b810 tx=0x7f2be0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:20.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.633+0000 7f2beeffd640 1 -- 172.21.15.106:0/2338455627 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2be40c1030 con 0x7f2bf0145030 2024-04-25T11:19:20.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.813+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bf01454a0 con 0x7f2bf0145030 2024-04-25T11:19:20.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.813+0000 7f2beeffd640 1 -- 172.21.15.106:0/2338455627 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f2bf01400d0 con 0x7f2bf0145030 2024-04-25T11:19:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:20.813 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T11:19:20.813 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 2 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2bdc06f400 msgr2=0x7f2bdc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2bdc06f400 0x7f2bdc071910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2be000b810 tx=0x7f2be0008040 comp rx=0 tx=0).stop 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf0145030 msgr2=0x7f2bf01f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf0145030 0x7f2bf01f70e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c240 tx=0x7f2bd8005840 comp rx=0 tx=0).stop 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 shutdown_connections 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf013f780 0x7f2bf013fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2bdc06f400 0x7f2bdc071910 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf0145030 0x7f2bf01f70e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 --2- 172.21.15.106:0/2338455627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bd8004810 0x7f2bf0147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:20.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 >> 172.21.15.106:0/2338455627 conn(0x7f2bf006c7c0 msgr2=0x7f2bf0142940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:20.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 shutdown_connections 2024-04-25T11:19:20.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:20.817+0000 7f2bf8488640 1 -- 172.21.15.106:0/2338455627 wait complete. 2024-04-25T11:19:22.833 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:22.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe71ca7640 1 Processor -- start 2024-04-25T11:19:22.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe71ca7640 1 -- start start 2024-04-25T11:19:22.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe71ca7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efe6c160f10 0x7efe6c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe71ca7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c15d6f0 0x7efe6c15db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe71ca7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efe6c0b43f0 0x7efe6c0b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe71ca7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efe6c0b5cc0 con 0x7efe6c0b43f0 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe71ca7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efe6c15cf50 con 0x7efe6c160f10 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe71ca7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c0b5f80 con 0x7efe6c15d6f0 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe6bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efe6c0b43f0 0x7efe6c0b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe6bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efe6c0b43f0 0x7efe6c0b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47968/0 (socket says 172.21.15.106:47968) 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe6bfff640 1 -- 172.21.15.106:0/367140288 learned_addr learned my addr 172.21.15.106:0/367140288 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe6b7fe640 1 --2- 172.21.15.106:0/367140288 >> v2:172.21.15.134:3300/0 conn(0x7efe6c160f10 0x7efe6c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.993+0000 7efe6affd640 1 --2- 172.21.15.106:0/367140288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c15d6f0 0x7efe6c15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6bfff640 1 -- 172.21.15.106:0/367140288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c15d6f0 msgr2=0x7efe6c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:22.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6bfff640 1 --2- 172.21.15.106:0/367140288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c15d6f0 0x7efe6c15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6bfff640 1 -- 172.21.15.106:0/367140288 >> v2:172.21.15.134:3300/0 conn(0x7efe6c160f10 msgr2=0x7efe6c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:22.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6bfff640 1 --2- 172.21.15.106:0/367140288 >> v2:172.21.15.134:3300/0 conn(0x7efe6c160f10 0x7efe6c15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6bfff640 1 -- 172.21.15.106:0/367140288 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c0b6260 con 0x7efe6c0b43f0 2024-04-25T11:19:22.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6bfff640 1 --2- 172.21.15.106:0/367140288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c0b43f0 0x7efe6c0b4800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efe6c072220 tx=0x7efe6401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ce07b333807f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:22.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6a7fc640 1 -- 172.21.15.106:0/367140288 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640a57b0 con 0x7efe6c0b43f0 2024-04-25T11:19:22.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6a7fc640 1 -- 172.21.15.106:0/367140288 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe640a5950 con 0x7efe6c0b43f0 2024-04-25T11:19:22.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe6a7fc640 1 -- 172.21.15.106:0/367140288 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640a5d20 con 0x7efe6c0b43f0 2024-04-25T11:19:22.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 -- 172.21.15.106:0/367140288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c0b43f0 msgr2=0x7efe6c0b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 --2- 172.21.15.106:0/367140288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c0b43f0 0x7efe6c0b4800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efe6c072220 tx=0x7efe6401c930 comp rx=0 tx=0).stop 2024-04-25T11:19:22.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 -- 172.21.15.106:0/367140288 shutdown_connections 2024-04-25T11:19:22.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 --2- 172.21.15.106:0/367140288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c0b43f0 0x7efe6c0b4800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 --2- 172.21.15.106:0/367140288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c15d6f0 0x7efe6c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 --2- 172.21.15.106:0/367140288 >> v2:172.21.15.134:3300/0 conn(0x7efe6c160f10 0x7efe6c15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 -- 172.21.15.106:0/367140288 >> 172.21.15.106:0/367140288 conn(0x7efe6c06c7c0 msgr2=0x7efe6c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:22.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 -- 172.21.15.106:0/367140288 shutdown_connections 2024-04-25T11:19:22.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:22.997+0000 7efe71ca7640 1 -- 172.21.15.106:0/367140288 wait complete. 2024-04-25T11:19:22.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 Processor -- start 2024-04-25T11:19:22.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 -- start start 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c0b43f0 0x7efe6c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6c15d6f0 0x7efe6c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c160f10 0x7efe6c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c0d1990 con 0x7efe6c160f10 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c0b4d40 con 0x7efe6c15d6f0 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c0d1b10 con 0x7efe6c0b43f0 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c160f10 0x7efe6c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c160f10 0x7efe6c154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47972/0 (socket says 172.21.15.106:47972) 2024-04-25T11:19:23.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 -- 172.21.15.106:0/105149551 learned_addr learned my addr 172.21.15.106:0/105149551 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6b7fe640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c0b43f0 0x7efe6c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6affd640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6c15d6f0 0x7efe6c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 -- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c0b43f0 msgr2=0x7efe6c140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c0b43f0 0x7efe6c140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 -- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6c15d6f0 msgr2=0x7efe6c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6c15d6f0 0x7efe6c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 -- 172.21.15.106:0/105149551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c15de50 con 0x7efe6c160f10 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6b7fe640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c0b43f0 0x7efe6c140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6affd640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6c15d6f0 0x7efe6c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe6bfff640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c160f10 0x7efe6c154f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe6401afe0 tx=0x7efe640196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe53fff640 1 -- 172.21.15.106:0/105149551 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640bc020 con 0x7efe6c160f10 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe53fff640 1 -- 172.21.15.106:0/105149551 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe640b8670 con 0x7efe6c160f10 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c1f9570 con 0x7efe6c160f10 2024-04-25T11:19:23.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe53fff640 1 -- 172.21.15.106:0/105149551 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64009510 con 0x7efe6c160f10 2024-04-25T11:19:23.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.001+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1f9b30 con 0x7efe6c160f10 2024-04-25T11:19:23.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.005+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6c156460 con 0x7efe6c160f10 2024-04-25T11:19:23.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.005+0000 7efe53fff640 1 -- 172.21.15.106:0/105149551 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe640bf020 con 0x7efe6c160f10 2024-04-25T11:19:23.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.005+0000 7efe53fff640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe5806f400 0x7efe58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.005+0000 7efe53fff640 1 -- 172.21.15.106:0/105149551 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7efe64129e60 con 0x7efe6c160f10 2024-04-25T11:19:23.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.005+0000 7efe6b7fe640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe5806f400 0x7efe58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.005+0000 7efe6b7fe640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe5806f400 0x7efe58071910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0d2420 tx=0x7efe5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.009+0000 7efe53fff640 1 -- 172.21.15.106:0/105149551 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe640c6030 con 0x7efe6c160f10 2024-04-25T11:19:23.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.205+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe6c0b4930 con 0x7efe6c160f10 2024-04-25T11:19:23.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.205+0000 7efe53fff640 1 -- 172.21.15.106:0/105149551 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7efe640165e0 con 0x7efe6c160f10 2024-04-25T11:19:23.205 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:23.205 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:19:23.206 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 3 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe5806f400 msgr2=0x7efe58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe5806f400 0x7efe58071910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0d2420 tx=0x7efe5c008040 comp rx=0 tx=0).stop 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c160f10 msgr2=0x7efe6c154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c160f10 0x7efe6c154f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe6401afe0 tx=0x7efe640196d0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 shutdown_connections 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe5806f400 0x7efe58071910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe6c160f10 0x7efe6c154f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe6c15d6f0 0x7efe6c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 --2- 172.21.15.106:0/105149551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe6c0b43f0 0x7efe6c140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 >> 172.21.15.106:0/105149551 conn(0x7efe6c06c7c0 msgr2=0x7efe6c0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 shutdown_connections 2024-04-25T11:19:23.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.209+0000 7efe71ca7640 1 -- 172.21.15.106:0/105149551 wait complete. 2024-04-25T11:19:23.227 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:23.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3c4da7640 1 Processor -- start 2024-04-25T11:19:23.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3c4da7640 1 -- start start 2024-04-25T11:19:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3c4da7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc3c00d2160 0x7fc3c00d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3c4da7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3c4da7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c00d2ab0 con 0x7fc3c00ba970 2024-04-25T11:19:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3c4da7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c00d1100 con 0x7fc3c00d2160 2024-04-25T11:19:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3c4da7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c00d0530 con 0x7fc3c00b5120 2024-04-25T11:19:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3bed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3bdd9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc3c00d2160 0x7fc3c00d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3bed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c00b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49056/0 (socket says 172.21.15.106:49056) 2024-04-25T11:19:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.377+0000 7fc3bed9d640 1 -- 172.21.15.106:0/2731843738 learned_addr learned my addr 172.21.15.106:0/2731843738 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bdd9b640 1 -- 172.21.15.106:0/2731843738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 msgr2=0x7fc3c00b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bdd9b640 1 --2- 172.21.15.106:0/2731843738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c00b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bdd9b640 1 -- 172.21.15.106:0/2731843738 >> v1:172.21.15.134:6789/0 conn(0x7fc3c00ba970 legacy=0x7fc3c00d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bdd9b640 1 -- 172.21.15.106:0/2731843738 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c00d07d0 con 0x7fc3c00d2160 2024-04-25T11:19:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bed9d640 1 --2- 172.21.15.106:0/2731843738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c00b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:23.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bdd9b640 1 --2- 172.21.15.106:0/2731843738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 0x7fc3c00d2570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40097a0 tx=0x7fc3b401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33a909efeef367aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bd59a640 1 -- 172.21.15.106:0/2731843738 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40b0070 con 0x7fc3c00d2160 2024-04-25T11:19:23.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bd59a640 1 -- 172.21.15.106:0/2731843738 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b40ab600 con 0x7fc3c00d2160 2024-04-25T11:19:23.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3bd59a640 1 -- 172.21.15.106:0/2731843738 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40ab9d0 con 0x7fc3c00d2160 2024-04-25T11:19:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3c4da7640 1 -- 172.21.15.106:0/2731843738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 msgr2=0x7fc3c00d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/2731843738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 0x7fc3c00d2570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40097a0 tx=0x7fc3b401c3f0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3c4da7640 1 -- 172.21.15.106:0/2731843738 shutdown_connections 2024-04-25T11:19:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/2731843738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c00b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/2731843738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 0x7fc3c00d2570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3c4da7640 1 -- 172.21.15.106:0/2731843738 >> 172.21.15.106:0/2731843738 conn(0x7fc3c00d57e0 msgr2=0x7fc3c00d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3c4da7640 1 -- 172.21.15.106:0/2731843738 shutdown_connections 2024-04-25T11:19:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.381+0000 7fc3c4da7640 1 -- 172.21.15.106:0/2731843738 wait complete. 2024-04-25T11:19:23.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 Processor -- start 2024-04-25T11:19:23.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 -- start start 2024-04-25T11:19:23.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c0162580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c00ba970 0x7fc3c015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 0x7fc3c00b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c00bb080 con 0x7fc3c00ba970 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01605b0 con 0x7fc3c00d2160 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c00bb200 con 0x7fc3c00b5120 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c00ba970 0x7fc3c015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c00ba970 0x7fc3c015cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47974/0 (socket says 172.21.15.106:47974) 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 -- 172.21.15.106:0/701718828 learned_addr learned my addr 172.21.15.106:0/701718828 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bed9d640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 0x7fc3c00b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3be59c640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c0162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 -- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 msgr2=0x7fc3c0162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c0162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 -- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 msgr2=0x7fc3c00b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 0x7fc3c00b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 -- 172.21.15.106:0/701718828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c0162ac0 con 0x7fc3c00ba970 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3be59c640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c0162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3bdd9b640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c00ba970 0x7fc3c015cf70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3b401a020 tx=0x7fc3b4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3aeffd640 1 -- 172.21.15.106:0/701718828 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40b0040 con 0x7fc3c00ba970 2024-04-25T11:19:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3aeffd640 1 -- 172.21.15.106:0/701718828 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b40bb070 con 0x7fc3c00ba970 2024-04-25T11:19:23.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c01622f0 con 0x7fc3c00ba970 2024-04-25T11:19:23.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3aeffd640 1 -- 172.21.15.106:0/701718828 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4006dc0 con 0x7fc3c00ba970 2024-04-25T11:19:23.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.385+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c015d4b0 con 0x7fc3c00ba970 2024-04-25T11:19:23.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.389+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c00d48d0 con 0x7fc3c00ba970 2024-04-25T11:19:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.389+0000 7fc3aeffd640 1 -- 172.21.15.106:0/701718828 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc3b40b3070 con 0x7fc3c00ba970 2024-04-25T11:19:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.389+0000 7fc3aeffd640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc3a006f400 0x7fc3a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.389+0000 7fc3be59c640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc3a006f400 0x7fc3a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.389+0000 7fc3aeffd640 1 -- 172.21.15.106:0/701718828 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc3b4128ae0 con 0x7fc3c00ba970 2024-04-25T11:19:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.389+0000 7fc3be59c640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc3a006f400 0x7fc3a0071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00b8320 tx=0x7fc3b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.393+0000 7fc3aeffd640 1 -- 172.21.15.106:0/701718828 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc3c01622f0 con 0x7fc3c00ba970 2024-04-25T11:19:23.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3c00d2ed0 con 0x7fc3c00ba970 2024-04-25T11:19:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3aeffd640 1 -- 172.21.15.106:0/701718828 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7fc3b4016070 con 0x7fc3c00ba970 2024-04-25T11:19:23.556 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:23.556 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:16.328802+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc3a006f400 msgr2=0x7fc3a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc3a006f400 0x7fc3a0071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00b8320 tx=0x7fc3b0009040 comp rx=0 tx=0).stop 2024-04-25T11:19:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c00ba970 msgr2=0x7fc3c015cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c00ba970 0x7fc3c015cf70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3b401a020 tx=0x7fc3b4019190 comp rx=0 tx=0).stop 2024-04-25T11:19:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 shutdown_connections 2024-04-25T11:19:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c00d2160 0x7fc3c00b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc3a006f400 0x7fc3a0071910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c00ba970 0x7fc3c015cf70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 --2- 172.21.15.106:0/701718828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3c00b5120 0x7fc3c0162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 >> 172.21.15.106:0/701718828 conn(0x7fc3c00d57e0 msgr2=0x7fc3c00b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 shutdown_connections 2024-04-25T11:19:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.557+0000 7fc3c4da7640 1 -- 172.21.15.106:0/701718828 wait complete. 2024-04-25T11:19:23.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:23.577 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:23.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f68672ec640 1 Processor -- start 2024-04-25T11:19:23.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f68672ec640 1 -- start start 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f68672ec640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686006f660 0x7f686006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f68672ec640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f68601678c0 0x7f6860169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f68672ec640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686006fcd0 con 0x7f686006f250 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f68672ec640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f686006fa70 con 0x7f68601678c0 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f68672ec640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f686006ffb0 con 0x7f686006f660 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865088640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f686006f250 0x7f686006c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59498/0 (socket says 172.21.15.106:59498) 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865088640 1 -- 172.21.15.106:0/3366772811 learned_addr learned my addr 172.21.15.106:0/3366772811 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865889640 1 --2- 172.21.15.106:0/3366772811 >> v2:172.21.15.134:3300/0 conn(0x7f68601678c0 0x7f6860169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6864887640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686006f660 0x7f686006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 634709031 0 0) 0x7f686006fcd0 con 0x7f686006f250 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f684c003530 con 0x7f686006f250 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3735107171 0 0) 0x7f684c003530 con 0x7f686006f250 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686006f660 msgr2=0x7f686006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686006f660 0x7f686006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 >> v2:172.21.15.134:3300/0 conn(0x7f68601678c0 msgr2=0x7f6860169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 --2- 172.21.15.106:0/3366772811 >> v2:172.21.15.134:3300/0 conn(0x7f68601678c0 0x7f6860169ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6860070290 con 0x7f686006f250 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f685c0016a0 con 0x7f686006f250 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 >> v1:172.21.15.106:6789/0 conn(0x7f686006f250 legacy=0x7f686006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f684c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f684c004810 0x7f684c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686006f660 0x7f684c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f686006fa70 con 0x7f684c0039c0 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f686006ffb0 con 0x7f684c004810 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f684c00ba80 con 0x7f686006f660 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865088640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f684c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6864887640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f684c004810 0x7f684c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865889640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686006f660 0x7f684c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865088640 1 -- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686006f660 msgr2=0x7f684c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865088640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686006f660 0x7f684c005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865088640 1 -- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f684c004810 msgr2=0x7f684c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865088640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f684c004810 0x7f684c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6865088640 1 -- 172.21.15.106:0/3366772811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c003530 con 0x7f684c0039c0 2024-04-25T11:19:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.733+0000 7f6864887640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f684c004810 0x7f684c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:23.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f6865088640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f684c003e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6860068aa0 tx=0x7f685c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab04bd81dc5b0a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c007d60 con 0x7f684c0039c0 2024-04-25T11:19:23.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c0a9020 con 0x7f684c0039c0 2024-04-25T11:19:23.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f6857fff640 1 -- 172.21.15.106:0/3366772811 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c005b00 con 0x7f684c0039c0 2024-04-25T11:19:23.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f68672ec640 1 -- 172.21.15.106:0/3366772811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 msgr2=0x7f684c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f68672ec640 1 --2- 172.21.15.106:0/3366772811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f684c003e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6860068aa0 tx=0x7f685c01cc10 comp rx=0 tx=0).stop 2024-04-25T11:19:23.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f6865889640 1 -- 172.21.15.106:0/3366772811 reap_dead start 2024-04-25T11:19:23.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f68672ec640 1 -- 172.21.15.106:0/3366772811 shutdown_connections 2024-04-25T11:19:23.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f68672ec640 1 -- 172.21.15.106:0/3366772811 >> 172.21.15.106:0/3366772811 conn(0x7f68600d1e30 msgr2=0x7f68600d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:23.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f68672ec640 1 -- 172.21.15.106:0/3366772811 shutdown_connections 2024-04-25T11:19:23.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.737+0000 7f68672ec640 1 -- 172.21.15.106:0/3366772811 wait complete. 2024-04-25T11:19:23.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 Processor -- start 2024-04-25T11:19:23.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 -- start start 2024-04-25T11:19:23.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f68600d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686015f480 0x7f68600d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f686015eea0 0x7f6860161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f686016a3a0 con 0x7f684c0039c0 2024-04-25T11:19:23.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f686016a220 con 0x7f686015eea0 2024-04-25T11:19:23.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f686016a520 con 0x7f686015f480 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f68600d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f68600d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48000/0 (socket says 172.21.15.106:48000) 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 -- 172.21.15.106:0/695560202 learned_addr learned my addr 172.21.15.106:0/695560202 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6864887640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686015f480 0x7f68600d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865889640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f686015eea0 0x7f6860161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 -- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686015f480 msgr2=0x7f68600d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686015f480 0x7f68600d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 -- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f686015eea0 msgr2=0x7f6860161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f686015eea0 0x7f6860161ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 -- 172.21.15.106:0/695560202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6860140110 con 0x7f684c0039c0 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865889640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f686015eea0 0x7f6860161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6864887640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686015f480 0x7f68600d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6865088640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f68600d0980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f685c01ca40 tx=0x7f684c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6855ffb640 1 -- 172.21.15.106:0/695560202 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c001ca0 con 0x7f684c0039c0 2024-04-25T11:19:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6855ffb640 1 -- 172.21.15.106:0/695560202 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c0a5a10 con 0x7f684c0039c0 2024-04-25T11:19:23.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6860160ee0 con 0x7f684c0039c0 2024-04-25T11:19:23.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f6855ffb640 1 -- 172.21.15.106:0/695560202 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c0a5dc0 con 0x7f684c0039c0 2024-04-25T11:19:23.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.741+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68601614a0 con 0x7f684c0039c0 2024-04-25T11:19:23.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.745+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6860050750 con 0x7f684c0039c0 2024-04-25T11:19:23.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.745+0000 7f6855ffb640 1 -- 172.21.15.106:0/695560202 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f685c0047f0 con 0x7f684c0039c0 2024-04-25T11:19:23.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.745+0000 7f6855ffb640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f684406f400 0x7f6844071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.745+0000 7f6855ffb640 1 -- 172.21.15.106:0/695560202 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f685c128500 con 0x7f684c0039c0 2024-04-25T11:19:23.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.745+0000 7f6864887640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f684406f400 0x7f6844071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.745+0000 7f6864887640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f684406f400 0x7f6844071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6850009b60 tx=0x7f6850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.749+0000 7f6855ffb640 1 -- 172.21.15.106:0/695560202 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f685c016950 con 0x7f684c0039c0 2024-04-25T11:19:23.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.909+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f686015fe70 con 0x7f684c0039c0 2024-04-25T11:19:23.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f6855ffb640 1 -- 172.21.15.106:0/695560202 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f685c0f57d0 con 0x7f684c0039c0 2024-04-25T11:19:23.910 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:19:23.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f684406f400 msgr2=0x7f6844071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f684406f400 0x7f6844071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6850009b60 tx=0x7f6850008040 comp rx=0 tx=0).stop 2024-04-25T11:19:23.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 msgr2=0x7f68600d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f68600d0980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f685c01ca40 tx=0x7f684c005840 comp rx=0 tx=0).stop 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 shutdown_connections 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f686015eea0 0x7f6860161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f686015f480 0x7f68600d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f684406f400 0x7f6844071910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 --2- 172.21.15.106:0/695560202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f684c0039c0 0x7f68600d0980 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.913+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 >> 172.21.15.106:0/695560202 conn(0x7f68600d1e30 msgr2=0x7f6860140bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.917+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 shutdown_connections 2024-04-25T11:19:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:23.917+0000 7f68672ec640 1 -- 172.21.15.106:0/695560202 wait complete. 2024-04-25T11:19:23.933 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:19:24.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.077+0000 7f17a4f0a640 1 Processor -- start 2024-04-25T11:19:24.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.077+0000 7f17a4f0a640 1 -- start start 2024-04-25T11:19:24.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f17a4f0a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f17a00b43f0 0x7f17a0072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f17a4f0a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a0072f90 0x7f17a0160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f17a4f0a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a00733a0 con 0x7f17a015c270 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f17a4f0a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f17a015fdb0 con 0x7f17a00b43f0 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f17a4f0a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17a00b49b0 con 0x7f17a0072f90 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f179dd9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f17a00b43f0 0x7f17a0072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f179e59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f17a015c270 0x7f17a00cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59502/0 (socket says 172.21.15.106:59502) 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f179e59c640 1 -- 172.21.15.106:0/1577065783 learned_addr learned my addr 172.21.15.106:0/1577065783 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f179ed9d640 1 --2- 172.21.15.106:0/1577065783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a0072f90 0x7f17a0160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f179dd9b640 1 -- 172.21.15.106:0/1577065783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a0072f90 msgr2=0x7f17a0160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f179dd9b640 1 --2- 172.21.15.106:0/1577065783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a0072f90 0x7f17a0160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f179dd9b640 1 -- 172.21.15.106:0/1577065783 >> v1:172.21.15.106:6789/0 conn(0x7f17a015c270 legacy=0x7f17a00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.081+0000 7f179dd9b640 1 -- 172.21.15.106:0/1577065783 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a00b4c90 con 0x7f17a00b43f0 2024-04-25T11:19:24.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f179dd9b640 1 --2- 172.21.15.106:0/1577065783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a00b43f0 0x7f17a0072a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17880097c0 tx=0x7f178801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969116b7c4931f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f179d59a640 1 -- 172.21.15.106:0/1577065783 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880b0070 con 0x7f17a00b43f0 2024-04-25T11:19:24.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f179d59a640 1 -- 172.21.15.106:0/1577065783 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17880ab600 con 0x7f17a00b43f0 2024-04-25T11:19:24.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f179d59a640 1 -- 172.21.15.106:0/1577065783 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880ab9d0 con 0x7f17a00b43f0 2024-04-25T11:19:24.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1577065783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a00b43f0 msgr2=0x7f17a0072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1577065783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a00b43f0 0x7f17a0072a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17880097c0 tx=0x7f178801c4a0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1577065783 shutdown_connections 2024-04-25T11:19:24.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1577065783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a0072f90 0x7f17a0160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1577065783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a00b43f0 0x7f17a0072a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1577065783 >> 172.21.15.106:0/1577065783 conn(0x7f17a006c7c0 msgr2=0x7f17a006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:24.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1577065783 shutdown_connections 2024-04-25T11:19:24.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1577065783 wait complete. 2024-04-25T11:19:24.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 Processor -- start 2024-04-25T11:19:24.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.085+0000 7f17a4f0a640 1 -- start start 2024-04-25T11:19:24.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f17a4f0a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17a0072f90 0x7f17a0165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f17a4f0a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a00b43f0 0x7f17a0168000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f17a4f0a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a015c270 0x7f17a0141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f17a4f0a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17a00cf3d0 con 0x7f17a015c270 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f17a4f0a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0073c00 con 0x7f17a0072f90 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f17a4f0a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17a0166e70 con 0x7f17a00b43f0 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a015c270 0x7f17a0141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a015c270 0x7f17a0141560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48014/0 (socket says 172.21.15.106:48014) 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 -- 172.21.15.106:0/1820225251 learned_addr learned my addr 172.21.15.106:0/1820225251 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179dd9b640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a00b43f0 0x7f17a0168000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179e59c640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17a0072f90 0x7f17a0165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 -- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a00b43f0 msgr2=0x7f17a0168000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a00b43f0 0x7f17a0168000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 -- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17a0072f90 msgr2=0x7f17a0165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17a0072f90 0x7f17a0165aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 -- 172.21.15.106:0/1820225251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a0156330 con 0x7f17a015c270 2024-04-25T11:19:24.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179dd9b640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a00b43f0 0x7f17a0168000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:24.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179e59c640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17a0072f90 0x7f17a0165aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:24.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f179ed9d640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a015c270 0x7f17a0141560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1790007d70 tx=0x7f179000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f1796ffd640 1 -- 172.21.15.106:0/1820225251 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790014420 con 0x7f17a015c270 2024-04-25T11:19:24.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f1796ffd640 1 -- 172.21.15.106:0/1820225251 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1790019070 con 0x7f17a015c270 2024-04-25T11:19:24.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f1796ffd640 1 -- 172.21.15.106:0/1820225251 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790014820 con 0x7f17a015c270 2024-04-25T11:19:24.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a00b1200 con 0x7f17a015c270 2024-04-25T11:19:24.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.089+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a00b17c0 con 0x7f17a015c270 2024-04-25T11:19:24.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.093+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a01560a0 con 0x7f17a015c270 2024-04-25T11:19:24.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.093+0000 7f1796ffd640 1 -- 172.21.15.106:0/1820225251 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17900149c0 con 0x7f17a015c270 2024-04-25T11:19:24.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.093+0000 7f1796ffd640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f178006f430 0x7f1780071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.093+0000 7f1796ffd640 1 -- 172.21.15.106:0/1820225251 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f179008f6e0 con 0x7f17a015c270 2024-04-25T11:19:24.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.093+0000 7f179e59c640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f178006f430 0x7f1780071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.093+0000 7f179e59c640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f178006f430 0x7f1780071940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1798007fd0 tx=0x7f1798009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.097+0000 7f1796ffd640 1 -- 172.21.15.106:0/1820225251 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f179005c9b0 con 0x7f17a015c270 2024-04-25T11:19:24.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f17a0166810 con 0x7f17a015c270 2024-04-25T11:19:24.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f1796ffd640 1 -- 172.21.15.106:0/1820225251 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f179005c2b0 con 0x7f17a015c270 2024-04-25T11:19:24.251 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:24.252 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-25T11:19:24.252 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:19:24.252 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-25T11:19:24.252 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-25T11:19:24.252 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:19:24.252 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f178006f430 msgr2=0x7f1780071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f178006f430 0x7f1780071940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1798007fd0 tx=0x7f1798009040 comp rx=0 tx=0).stop 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a015c270 msgr2=0x7f17a0141560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a015c270 0x7f17a0141560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1790007d70 tx=0x7f179000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 shutdown_connections 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f178006f430 0x7f1780071940 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a015c270 0x7f17a0141560 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17a00b43f0 0x7f17a0168000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 --2- 172.21.15.106:0/1820225251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17a0072f90 0x7f17a0165aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 >> 172.21.15.106:0/1820225251 conn(0x7f17a006c7c0 msgr2=0x7f17a00d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 shutdown_connections 2024-04-25T11:19:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.253+0000 7f17a4f0a640 1 -- 172.21.15.106:0/1820225251 wait complete. 2024-04-25T11:19:24.271 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:19:24.271 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:19:24.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3256c0640 1 Processor -- start 2024-04-25T11:19:24.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3256c0640 1 -- start start 2024-04-25T11:19:24.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3256c0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3200d26b0 0x7fa3200b7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3256c0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa3200b7870 0x7fa3200b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3256c0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d4300 0x7fa3200d4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3256c0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa3200b81c0 con 0x7fa3200d26b0 2024-04-25T11:19:24.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3256c0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa3201643e0 con 0x7fa3200b7870 2024-04-25T11:19:24.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3256c0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa3201612c0 con 0x7fa3200d4300 2024-04-25T11:19:24.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa31effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3200d26b0 0x7fa3200b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa31effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3200d26b0 0x7fa3200b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48016/0 (socket says 172.21.15.106:48016) 2024-04-25T11:19:24.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa31effd640 1 -- 172.21.15.106:0/1698955780 learned_addr learned my addr 172.21.15.106:0/1698955780 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:24.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa3167fc640 1 --2- 172.21.15.106:0/1698955780 >> v2:172.21.15.134:3300/0 conn(0x7fa3200b7870 0x7fa3200b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa31f7fe640 1 --2- 172.21.15.106:0/1698955780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d4300 0x7fa3200d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa31effd640 1 -- 172.21.15.106:0/1698955780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d4300 msgr2=0x7fa3200d4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:24.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa31effd640 1 --2- 172.21.15.106:0/1698955780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d4300 0x7fa3200d4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa31effd640 1 -- 172.21.15.106:0/1698955780 >> v2:172.21.15.134:3300/0 conn(0x7fa3200b7870 msgr2=0x7fa3200b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:24.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.413+0000 7fa31effd640 1 --2- 172.21.15.106:0/1698955780 >> v2:172.21.15.134:3300/0 conn(0x7fa3200b7870 0x7fa3200b7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa31effd640 1 -- 172.21.15.106:0/1698955780 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa320163ab0 con 0x7fa3200d26b0 2024-04-25T11:19:24.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa31effd640 1 --2- 172.21.15.106:0/1698955780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d26b0 0x7fa3200b7330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa32008ec40 tx=0x7fa31001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb4dc99b72e0879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa31e7fc640 1 -- 172.21.15.106:0/1698955780 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100aa610 con 0x7fa3200d26b0 2024-04-25T11:19:24.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa31e7fc640 1 -- 172.21.15.106:0/1698955780 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3100af070 con 0x7fa3200d26b0 2024-04-25T11:19:24.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa31e7fc640 1 -- 172.21.15.106:0/1698955780 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100aa9d0 con 0x7fa3200d26b0 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 -- 172.21.15.106:0/1698955780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d26b0 msgr2=0x7fa3200b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 --2- 172.21.15.106:0/1698955780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d26b0 0x7fa3200b7330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa32008ec40 tx=0x7fa31001c4f0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 -- 172.21.15.106:0/1698955780 shutdown_connections 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 --2- 172.21.15.106:0/1698955780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d4300 0x7fa3200d4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 --2- 172.21.15.106:0/1698955780 >> v2:172.21.15.134:3300/0 conn(0x7fa3200b7870 0x7fa3200b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 --2- 172.21.15.106:0/1698955780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d26b0 0x7fa3200b7330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 -- 172.21.15.106:0/1698955780 >> 172.21.15.106:0/1698955780 conn(0x7fa32006e5b0 msgr2=0x7fa320073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 -- 172.21.15.106:0/1698955780 shutdown_connections 2024-04-25T11:19:24.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.417+0000 7fa3256c0640 1 -- 172.21.15.106:0/1698955780 wait complete. 2024-04-25T11:19:24.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3256c0640 1 Processor -- start 2024-04-25T11:19:24.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3256c0640 1 -- start start 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3256c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3200b7870 0x7fa32014fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3256c0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d26b0 0x7fa320153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3256c0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d4300 0x7fa320153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3256c0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa3200d2c40 con 0x7fa3200d4300 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3256c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3200d2ac0 con 0x7fa3200b7870 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3256c0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa32015fdc0 con 0x7fa3200d26b0 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d4300 0x7fa320153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d4300 0x7fa320153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48020/0 (socket says 172.21.15.106:48020) 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa31f7fe640 1 -- 172.21.15.106:0/446535201 learned_addr learned my addr 172.21.15.106:0/446535201 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:24.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa31effd640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3200b7870 0x7fa32014fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.421+0000 7fa3167fc640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d26b0 0x7fa320153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa31f7fe640 1 -- 172.21.15.106:0/446535201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d26b0 msgr2=0x7fa320153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa31f7fe640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d26b0 0x7fa320153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa31f7fe640 1 -- 172.21.15.106:0/446535201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3200b7870 msgr2=0x7fa32014fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa31f7fe640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3200b7870 0x7fa32014fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa31f7fe640 1 -- 172.21.15.106:0/446535201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32016a630 con 0x7fa3200d4300 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa31f7fe640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d4300 0x7fa320153d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa318007d80 tx=0x7fa31800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa317fff640 1 -- 172.21.15.106:0/446535201 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318014420 con 0x7fa3200d4300 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa317fff640 1 -- 172.21.15.106:0/446535201 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa318019070 con 0x7fa3200d4300 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32016a910 con 0x7fa3200d4300 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa317fff640 1 -- 172.21.15.106:0/446535201 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318014800 con 0x7fa3200d4300 2024-04-25T11:19:24.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3200cee70 con 0x7fa3200d4300 2024-04-25T11:19:24.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32006ee30 con 0x7fa3200d4300 2024-04-25T11:19:24.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa317fff640 1 -- 172.21.15.106:0/446535201 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa3180149a0 con 0x7fa3200d4300 2024-04-25T11:19:24.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa317fff640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa30006f430 0x7fa300071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa317fff640 1 -- 172.21.15.106:0/446535201 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa31808f6c0 con 0x7fa3200d4300 2024-04-25T11:19:24.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa31effd640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa30006f430 0x7fa300071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.425+0000 7fa31effd640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa30006f430 0x7fa300071940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3100ae000 tx=0x7fa3100a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.429+0000 7fa317fff640 1 -- 172.21.15.106:0/446535201 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa31805c340 con 0x7fa3200d4300 2024-04-25T11:19:24.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:24.581+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa320050830 con 0x7fa3200d4300 2024-04-25T11:19:24.687 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:19:24.690 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:19:25.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.381+0000 7fa317fff640 1 -- 172.21.15.106:0/446535201 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fa318060640 con 0x7fa3200d4300 2024-04-25T11:19:25.381 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa30006f430 msgr2=0x7fa300071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa30006f430 0x7fa300071940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3100ae000 tx=0x7fa3100a7000 comp rx=0 tx=0).stop 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d4300 msgr2=0x7fa320153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d4300 0x7fa320153d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa318007d80 tx=0x7fa31800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 shutdown_connections 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa30006f430 0x7fa300071940 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3200d4300 0x7fa320153d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3200d26b0 0x7fa320153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 --2- 172.21.15.106:0/446535201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3200b7870 0x7fa32014fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 >> 172.21.15.106:0/446535201 conn(0x7fa32006e5b0 msgr2=0x7fa320155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:25.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 shutdown_connections 2024-04-25T11:19:25.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.385+0000 7fa3256c0640 1 -- 172.21.15.106:0/446535201 wait complete. 2024-04-25T11:19:25.411 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:19:25.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.569+0000 7f02211ea640 1 Processor -- start 2024-04-25T11:19:25.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.569+0000 7f02211ea640 1 -- start start 2024-04-25T11:19:25.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.569+0000 7f02211ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c0d1d00 0x7f021c161b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.569+0000 7f02211ea640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f021c072a20 0x7f021c072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.569+0000 7f02211ea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f021c15e120 con 0x7f021c161710 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.569+0000 7f02211ea640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f021c1621d0 con 0x7f021c072a20 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.569+0000 7f02211ea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f021c1624b0 con 0x7f021c0d1d00 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.569+0000 7f021b59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f021c072a20 0x7f021c072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021b59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f021c072a20 0x7f021c072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45894/0 (socket says 172.21.15.106:45894) 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021b59e640 1 -- 172.21.15.106:0/1445231650 learned_addr learned my addr 172.21.15.106:0/1445231650 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021a59c640 1 --2- 172.21.15.106:0/1445231650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c0d1d00 0x7f021c161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021b59e640 1 -- 172.21.15.106:0/1445231650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c0d1d00 msgr2=0x7f021c161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021b59e640 1 --2- 172.21.15.106:0/1445231650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c0d1d00 0x7f021c161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021b59e640 1 -- 172.21.15.106:0/1445231650 >> v1:172.21.15.134:6789/0 conn(0x7f021c161710 legacy=0x7f021c0d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021b59e640 1 -- 172.21.15.106:0/1445231650 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c15e3a0 con 0x7f021c072a20 2024-04-25T11:19:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021a59c640 1 --2- 172.21.15.106:0/1445231650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c0d1d00 0x7f021c161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f021b59e640 1 --2- 172.21.15.106:0/1445231650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c072a20 0x7f021c072e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0214009e50 tx=0x7f021401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21832eebd1177e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f0219d9b640 1 -- 172.21.15.106:0/1445231650 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02140a5800 con 0x7f021c072a20 2024-04-25T11:19:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f0219d9b640 1 -- 172.21.15.106:0/1445231650 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02140a59a0 con 0x7f021c072a20 2024-04-25T11:19:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f0219d9b640 1 -- 172.21.15.106:0/1445231650 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02140a5d70 con 0x7f021c072a20 2024-04-25T11:19:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f02211ea640 1 -- 172.21.15.106:0/1445231650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c072a20 msgr2=0x7f021c072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f02211ea640 1 --2- 172.21.15.106:0/1445231650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c072a20 0x7f021c072e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0214009e50 tx=0x7f021401ca00 comp rx=0 tx=0).stop 2024-04-25T11:19:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f02211ea640 1 -- 172.21.15.106:0/1445231650 shutdown_connections 2024-04-25T11:19:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f02211ea640 1 --2- 172.21.15.106:0/1445231650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c072a20 0x7f021c072e30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f02211ea640 1 --2- 172.21.15.106:0/1445231650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c0d1d00 0x7f021c161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f02211ea640 1 -- 172.21.15.106:0/1445231650 >> 172.21.15.106:0/1445231650 conn(0x7f021c06c7c0 msgr2=0x7f021c06cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f02211ea640 1 -- 172.21.15.106:0/1445231650 shutdown_connections 2024-04-25T11:19:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.573+0000 7f02211ea640 1 -- 172.21.15.106:0/1445231650 wait complete. 2024-04-25T11:19:25.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02211ea640 1 Processor -- start 2024-04-25T11:19:25.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02211ea640 1 -- start start 2024-04-25T11:19:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02211ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c072a20 0x7f021c14a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02211ea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c161710 0x7f021c0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02211ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c06ea40 0x7f021c155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02211ea640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f021c0cf140 con 0x7f021c161710 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02211ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f021c0cefc0 con 0x7f021c072a20 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02211ea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f021c16b2c0 con 0x7f021c06ea40 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c161710 0x7f021c0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c161710 0x7f021c0705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45906/0 (socket says 172.21.15.106:45906) 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 -- 172.21.15.106:0/1850616353 learned_addr learned my addr 172.21.15.106:0/1850616353 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021b59e640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c06ea40 0x7f021c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021ad9d640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c072a20 0x7f021c14a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 -- 172.21.15.106:0/1850616353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c06ea40 msgr2=0x7f021c155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c06ea40 0x7f021c155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 -- 172.21.15.106:0/1850616353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c072a20 msgr2=0x7f021c14a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c072a20 0x7f021c14a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 -- 172.21.15.106:0/1850616353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c06f520 con 0x7f021c161710 2024-04-25T11:19:25.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f021a59c640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c161710 0x7f021c0705c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f021000d830 tx=0x7f021000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02037fe640 1 -- 172.21.15.106:0/1850616353 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210018020 con 0x7f021c161710 2024-04-25T11:19:25.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02037fe640 1 -- 172.21.15.106:0/1850616353 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02100096c0 con 0x7f021c161710 2024-04-25T11:19:25.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.577+0000 7f02037fe640 1 -- 172.21.15.106:0/1850616353 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210009ab0 con 0x7f021c161710 2024-04-25T11:19:25.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.581+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c06f800 con 0x7f021c161710 2024-04-25T11:19:25.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.581+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c06fdc0 con 0x7f021c161710 2024-04-25T11:19:25.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.581+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021c070b00 con 0x7f021c161710 2024-04-25T11:19:25.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.581+0000 7f02037fe640 1 -- 172.21.15.106:0/1850616353 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f021001b020 con 0x7f021c161710 2024-04-25T11:19:25.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.581+0000 7f02037fe640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f020806f430 0x7f0208071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.581+0000 7f02037fe640 1 -- 172.21.15.106:0/1850616353 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f021008fb60 con 0x7f021c161710 2024-04-25T11:19:25.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.581+0000 7f021ad9d640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f020806f430 0x7f0208071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.581+0000 7f021ad9d640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f020806f430 0x7f0208071940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f020c0015b0 tx=0x7f020c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.585+0000 7f02037fe640 1 -- 172.21.15.106:0/1850616353 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0210022020 con 0x7f021c161710 2024-04-25T11:19:25.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:25.749+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f021c06f2b0 con 0x7f021c161710 2024-04-25T11:19:26.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.397+0000 7f02037fe640 1 -- 172.21.15.106:0/1850616353 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f021005cda0 con 0x7f021c161710 2024-04-25T11:19:26.396 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f020806f430 msgr2=0x7f0208071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f020806f430 0x7f0208071940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f020c0015b0 tx=0x7f020c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c161710 msgr2=0x7f021c0705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c161710 0x7f021c0705c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f021000d830 tx=0x7f021000dcf0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 shutdown_connections 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f021c06ea40 0x7f021c155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f020806f430 0x7f0208071940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c161710 0x7f021c0705c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 --2- 172.21.15.106:0/1850616353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c072a20 0x7f021c14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 >> 172.21.15.106:0/1850616353 conn(0x7f021c06c7c0 msgr2=0x7f021c1600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 shutdown_connections 2024-04-25T11:19:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.401+0000 7f02211ea640 1 -- 172.21.15.106:0/1850616353 wait complete. 2024-04-25T11:19:26.437 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:19:26.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa7d30d640 1 Processor -- start 2024-04-25T11:19:26.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa7d30d640 1 -- start start 2024-04-25T11:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa7d30d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa780d2150 0x7ffa780d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa7d30d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 0x7ffa780d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa7d30d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa780d1350 con 0x7ffa7815fd10 2024-04-25T11:19:26.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa7d30d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffa780d2aa0 con 0x7ffa780d2150 2024-04-25T11:19:26.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa7d30d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa78163cc0 con 0x7ffa780d44a0 2024-04-25T11:19:26.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa76ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ffa7815fd10 0x7ffa780b8d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56904/0 (socket says 172.21.15.106:56904) 2024-04-25T11:19:26.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa76ffd640 1 -- 172.21.15.106:0/3805978815 learned_addr learned my addr 172.21.15.106:0/3805978815 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:26.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa767fc640 1 --2- 172.21.15.106:0/3805978815 >> v2:172.21.15.134:3300/0 conn(0x7ffa780d2150 0x7ffa780d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.589+0000 7ffa777fe640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 0x7ffa780d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2996605083 0 0) 0x7ffa780d1350 con 0x7ffa7815fd10 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa60003530 con 0x7ffa7815fd10 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2729032464 0 0) 0x7ffa60003530 con 0x7ffa7815fd10 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 msgr2=0x7ffa780d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 0x7ffa780d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 >> v2:172.21.15.134:3300/0 conn(0x7ffa780d2150 msgr2=0x7ffa780d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 --2- 172.21.15.106:0/3805978815 >> v2:172.21.15.134:3300/0 conn(0x7ffa780d2150 0x7ffa780d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa78163fa0 con 0x7ffa7815fd10 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7ffa68001680 con 0x7ffa7815fd10 2024-04-25T11:19:26.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 >> v1:172.21.15.106:6789/0 conn(0x7ffa7815fd10 legacy=0x7ffa780b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 0x7ffa600039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa60003f40 0x7ffa60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa60004d50 0x7ffa60005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780d2aa0 con 0x7ffa60004d50 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78163cc0 con 0x7ffa60003f40 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6000ba20 con 0x7ffa780d44a0 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa777fe640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa60004d50 0x7ffa60005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa76ffd640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 0x7ffa600039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa767fc640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa60003f40 0x7ffa60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa777fe640 1 -- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 msgr2=0x7ffa600039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:26.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa777fe640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 0x7ffa600039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa777fe640 1 -- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa60003f40 msgr2=0x7ffa60004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:26.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa777fe640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa60003f40 0x7ffa60004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa777fe640 1 -- 172.21.15.106:0/3805978815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa60003530 con 0x7ffa60004d50 2024-04-25T11:19:26.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa76ffd640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780d44a0 0x7ffa600039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:26.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa777fe640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa60004d50 0x7ffa60005160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa7808ea80 tx=0x7ffa7001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918a39174d2de820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:26.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.593+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700b2070 con 0x7ffa60004d50 2024-04-25T11:19:26.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa700a5c50 con 0x7ffa60004d50 2024-04-25T11:19:26.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa75ffb640 1 -- 172.21.15.106:0/3805978815 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70009c10 con 0x7ffa60004d50 2024-04-25T11:19:26.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3805978815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa60004d50 msgr2=0x7ffa60005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa7d30d640 1 --2- 172.21.15.106:0/3805978815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa60004d50 0x7ffa60005160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa7808ea80 tx=0x7ffa7001cc40 comp rx=0 tx=0).stop 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa777fe640 1 -- 172.21.15.106:0/3805978815 reap_dead start 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3805978815 shutdown_connections 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3805978815 >> 172.21.15.106:0/3805978815 conn(0x7ffa7806d7f0 msgr2=0x7ffa7806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3805978815 shutdown_connections 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3805978815 wait complete. 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa7d30d640 1 Processor -- start 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.597+0000 7ffa7d30d640 1 -- start start 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780ca150 0x7ffa7814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa78150b60 0x7ffa78150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa781514b0 0x7ffa781f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78160120 con 0x7ffa78150b60 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78164560 con 0x7ffa781514b0 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa781638e0 con 0x7ffa780ca150 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa78150b60 0x7ffa78150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa78150b60 0x7ffa78150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45914/0 (socket says 172.21.15.106:45914) 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 -- 172.21.15.106:0/3694075719 learned_addr learned my addr 172.21.15.106:0/3694075719 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa777fe640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa781514b0 0x7ffa781f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa76ffd640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780ca150 0x7ffa7814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 -- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780ca150 msgr2=0x7ffa7814e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780ca150 0x7ffa7814e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 -- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa781514b0 msgr2=0x7ffa781f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa781514b0 0x7ffa781f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 -- 172.21.15.106:0/3694075719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa780ca560 con 0x7ffa78150b60 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa777fe640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa781514b0 0x7ffa781f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa767fc640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa78150b60 0x7ffa78150f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c011600 tx=0x7ffa6c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa5f7fe640 1 -- 172.21.15.106:0/3694075719 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c00f920 con 0x7ffa78150b60 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa5f7fe640 1 -- 172.21.15.106:0/3694075719 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c00fac0 con 0x7ffa78150b60 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa780ca830 con 0x7ffa78150b60 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa5f7fe640 1 -- 172.21.15.106:0/3694075719 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c00fe70 con 0x7ffa78150b60 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa780cadf0 con 0x7ffa78150b60 2024-04-25T11:19:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78050750 con 0x7ffa78150b60 2024-04-25T11:19:26.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa5f7fe640 1 -- 172.21.15.106:0/3694075719 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffa6c015460 con 0x7ffa78150b60 2024-04-25T11:19:26.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa5f7fe640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffa6406f400 0x7ffa64071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa76ffd640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffa6406f400 0x7ffa64071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.601+0000 7ffa5f7fe640 1 -- 172.21.15.106:0/3694075719 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7ffa6c08fb00 con 0x7ffa78150b60 2024-04-25T11:19:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.605+0000 7ffa76ffd640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffa6406f400 0x7ffa64071910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa68000fc0 tx=0x7ffa68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:26.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.609+0000 7ffa5f7fe640 1 -- 172.21.15.106:0/3694075719 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa6c019030 con 0x7ffa78150b60 2024-04-25T11:19:26.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:26.801+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ffa780d3be0 con 0x7ffa78150b60 2024-04-25T11:19:27.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.409+0000 7ffa5f7fe640 1 -- 172.21.15.106:0/3694075719 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ffa6c05cbc0 con 0x7ffa78150b60 2024-04-25T11:19:27.412 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T11:19:27.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.425+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffa6406f400 msgr2=0x7ffa64071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.425+0000 7ffa7d30d640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffa6406f400 0x7ffa64071910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa68000fc0 tx=0x7ffa68009040 comp rx=0 tx=0).stop 2024-04-25T11:19:27.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.425+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa78150b60 msgr2=0x7ffa78150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.425+0000 7ffa7d30d640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa78150b60 0x7ffa78150f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c011600 tx=0x7ffa6c011ac0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.433+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 shutdown_connections 2024-04-25T11:19:27.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.433+0000 7ffa7d30d640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa781514b0 0x7ffa781f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.433+0000 7ffa7d30d640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffa6406f400 0x7ffa64071910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.433+0000 7ffa7d30d640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa78150b60 0x7ffa78150f70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.433+0000 7ffa7d30d640 1 --2- 172.21.15.106:0/3694075719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa780ca150 0x7ffa7814e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.433+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 >> 172.21.15.106:0/3694075719 conn(0x7ffa7806d7f0 msgr2=0x7ffa78162690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:27.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.433+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 shutdown_connections 2024-04-25T11:19:27.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.433+0000 7ffa7d30d640 1 -- 172.21.15.106:0/3694075719 wait complete. 2024-04-25T11:19:27.463 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff99a5cf640 1 Processor -- start 2024-04-25T11:19:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff99a5cf640 1 -- start start 2024-04-25T11:19:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff99a5cf640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff9940d1d00 0x7ff9940d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff99a5cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940d1190 0x7ff9940d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff99a5cf640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9940b2700 con 0x7ff9940b6410 2024-04-25T11:19:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff99a5cf640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff9940d0fc0 con 0x7ff9940d1d00 2024-04-25T11:19:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff99a5cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9940d0ce0 con 0x7ff9940d1190 2024-04-25T11:19:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff998b6c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff9940b6410 0x7ff994161dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56918/0 (socket says 172.21.15.106:56918) 2024-04-25T11:19:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff998b6c640 1 -- 172.21.15.106:0/1168332437 learned_addr learned my addr 172.21.15.106:0/1168332437 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff993fff640 1 --2- 172.21.15.106:0/1168332437 >> v2:172.21.15.134:3300/0 conn(0x7ff9940d1d00 0x7ff9940d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff9937fe640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940d1190 0x7ff9940d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384603032 0 0) 0x7ff9940b2700 con 0x7ff9940b6410 2024-04-25T11:19:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff980003530 con 0x7ff9940b6410 2024-04-25T11:19:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216999409 0 0) 0x7ff980003530 con 0x7ff9940b6410 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940d1190 msgr2=0x7ff9940d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940d1190 0x7ff9940d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 >> v2:172.21.15.134:3300/0 conn(0x7ff9940d1d00 msgr2=0x7ff9940d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 --2- 172.21.15.106:0/1168332437 >> v2:172.21.15.134:3300/0 conn(0x7ff9940d1d00 0x7ff9940d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9940b8d00 con 0x7ff9940b6410 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.609+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7ff984001680 con 0x7ff9940b6410 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 >> v1:172.21.15.106:6789/0 conn(0x7ff9940b6410 legacy=0x7ff994161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800039c0 0x7ff980003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff980004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940d1190 0x7ff980005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9940d0fc0 con 0x7ff980004810 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9940d0ce0 con 0x7ff9800039c0 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff98000b9d0 con 0x7ff9940d1190 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff9937fe640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff980004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff993fff640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800039c0 0x7ff980003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff9937fe640 1 -- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940d1190 msgr2=0x7ff980005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff9937fe640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940d1190 0x7ff980005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff9937fe640 1 -- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800039c0 msgr2=0x7ff980003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff9937fe640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800039c0 0x7ff980003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff9937fe640 1 -- 172.21.15.106:0/1168332437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff980003530 con 0x7ff980004810 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff9937fe640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff980004c20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009ab0 tx=0x7ff98c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611c787ba3e3b20e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0b2070 con 0x7ff980004810 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff98c0a5680 con 0x7ff980004810 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff992ffd640 1 -- 172.21.15.106:0/1168332437 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0096b0 con 0x7ff980004810 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff99a5cf640 1 -- 172.21.15.106:0/1168332437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 msgr2=0x7ff980004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff99a5cf640 1 --2- 172.21.15.106:0/1168332437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff980004c20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009ab0 tx=0x7ff98c01ca90 comp rx=0 tx=0).stop 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff998b6c640 1 -- 172.21.15.106:0/1168332437 reap_dead start 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff99a5cf640 1 -- 172.21.15.106:0/1168332437 shutdown_connections 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff99a5cf640 1 -- 172.21.15.106:0/1168332437 >> 172.21.15.106:0/1168332437 conn(0x7ff9940d0040 msgr2=0x7ff9940d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff99a5cf640 1 -- 172.21.15.106:0/1168332437 shutdown_connections 2024-04-25T11:19:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.613+0000 7ff99a5cf640 1 -- 172.21.15.106:0/1168332437 wait complete. 2024-04-25T11:19:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 Processor -- start 2024-04-25T11:19:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 -- start start 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff99409e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99415ebe0 0x7ff9940a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940a0950 0x7ff99409d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9940b8840 con 0x7ff980004810 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9940b86c0 con 0x7ff99415ebe0 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9940d15a0 con 0x7ff9940a0950 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff99409e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff99409e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45938/0 (socket says 172.21.15.106:45938) 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 -- 172.21.15.106:0/2102564130 learned_addr learned my addr 172.21.15.106:0/2102564130 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff9937fe640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99415ebe0 0x7ff9940a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff998b6c640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940a0950 0x7ff99409d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 -- 172.21.15.106:0/2102564130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940a0950 msgr2=0x7ff99409d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940a0950 0x7ff99409d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 -- 172.21.15.106:0/2102564130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99415ebe0 msgr2=0x7ff9940a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99415ebe0 0x7ff9940a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 -- 172.21.15.106:0/2102564130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9940a13b0 con 0x7ff980004810 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff993fff640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff99409e7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00f400 tx=0x7ff97c00f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff990ff9640 1 -- 172.21.15.106:0/2102564130 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0147e0 con 0x7ff980004810 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff990ff9640 1 -- 172.21.15.106:0/2102564130 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97c01b070 con 0x7ff980004810 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9940a10d0 con 0x7ff980004810 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.617+0000 7ff990ff9640 1 -- 172.21.15.106:0/2102564130 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c014be0 con 0x7ff980004810 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.621+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99415e3e0 con 0x7ff980004810 2024-04-25T11:19:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.621+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9940d2df0 con 0x7ff980004810 2024-04-25T11:19:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.621+0000 7ff990ff9640 1 -- 172.21.15.106:0/2102564130 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff97c029430 con 0x7ff980004810 2024-04-25T11:19:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.621+0000 7ff990ff9640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff97406f400 0x7ff974071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.621+0000 7ff990ff9640 1 -- 172.21.15.106:0/2102564130 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7ff97c090cc0 con 0x7ff980004810 2024-04-25T11:19:27.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.621+0000 7ff9937fe640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff97406f400 0x7ff974071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.621+0000 7ff9937fe640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff97406f400 0x7ff974071910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0af000 tx=0x7ff98c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.625+0000 7ff990ff9640 1 -- 172.21.15.106:0/2102564130 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9940a13b0 con 0x7ff980004810 2024-04-25T11:19:27.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.789+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9940ceeb0 con 0x7ff980004810 2024-04-25T11:19:27.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff990ff9640 1 -- 172.21.15.106:0/2102564130 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16654 (secure 0 0 0) 0x7ff97c05dd00 con 0x7ff980004810 2024-04-25T11:19:27.791 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:27.792 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":19,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:26.806248+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff97406f400 msgr2=0x7ff974071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff97406f400 0x7ff974071910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0af000 tx=0x7ff98c0046d0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 msgr2=0x7ff99409e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff99409e7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00f400 tx=0x7ff97c00f8c0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 shutdown_connections 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9940a0950 0x7ff99409d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99415ebe0 0x7ff9940a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff97406f400 0x7ff974071910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 --2- 172.21.15.106:0/2102564130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff980004810 0x7ff99409e7c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 >> 172.21.15.106:0/2102564130 conn(0x7ff9940d0040 msgr2=0x7ff9940b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:27.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 shutdown_connections 2024-04-25T11:19:27.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.793+0000 7ff99a5cf640 1 -- 172.21.15.106:0/2102564130 wait complete. 2024-04-25T11:19:27.813 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:27.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa003c11640 1 Processor -- start 2024-04-25T11:19:27.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa003c11640 1 -- start start 2024-04-25T11:19:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa003c11640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ffc160f10 0x7f9ffc15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa003c11640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc0b4b00 0x7f9ffc15db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa003c11640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ffc0b5cc0 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa003c11640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9ffc15cf50 con 0x7f9ffc160f10 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa003c11640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc0b5f80 con 0x7f9ffc0b4b00 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa0019ad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ffc160f10 0x7f9ffc15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa0011ac640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9ffc15d6f0 0x7f9ffc0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56924/0 (socket says 172.21.15.106:56924) 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa0011ac640 1 -- 172.21.15.106:0/3719591207 learned_addr learned my addr 172.21.15.106:0/3719591207 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa0021ae640 1 --2- 172.21.15.106:0/3719591207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc0b4b00 0x7f9ffc15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa0019ad640 1 -- 172.21.15.106:0/3719591207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc0b4b00 msgr2=0x7f9ffc15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa0019ad640 1 --2- 172.21.15.106:0/3719591207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc0b4b00 0x7f9ffc15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa0019ad640 1 -- 172.21.15.106:0/3719591207 >> v1:172.21.15.106:6789/0 conn(0x7f9ffc15d6f0 legacy=0x7f9ffc0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.961+0000 7fa0019ad640 1 -- 172.21.15.106:0/3719591207 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc0b6260 con 0x7f9ffc160f10 2024-04-25T11:19:27.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa0019ad640 1 --2- 172.21.15.106:0/3719591207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc160f10 0x7f9ffc15ca10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80097c0 tx=0x7f9ff801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d6443296d7f042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa0009ab640 1 -- 172.21.15.106:0/3719591207 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80b1070 con 0x7f9ffc160f10 2024-04-25T11:19:27.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa0009ab640 1 -- 172.21.15.106:0/3719591207 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff80a8050 con 0x7f9ffc160f10 2024-04-25T11:19:27.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa0009ab640 1 -- 172.21.15.106:0/3719591207 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80ac860 con 0x7f9ffc160f10 2024-04-25T11:19:27.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa003c11640 1 -- 172.21.15.106:0/3719591207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc160f10 msgr2=0x7f9ffc15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa003c11640 1 --2- 172.21.15.106:0/3719591207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc160f10 0x7f9ffc15ca10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80097c0 tx=0x7f9ff801c3e0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa003c11640 1 -- 172.21.15.106:0/3719591207 shutdown_connections 2024-04-25T11:19:27.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa003c11640 1 --2- 172.21.15.106:0/3719591207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc0b4b00 0x7f9ffc15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa003c11640 1 --2- 172.21.15.106:0/3719591207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc160f10 0x7f9ffc15ca10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa003c11640 1 -- 172.21.15.106:0/3719591207 >> 172.21.15.106:0/3719591207 conn(0x7f9ffc06c7c0 msgr2=0x7f9ffc06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:27.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa003c11640 1 -- 172.21.15.106:0/3719591207 shutdown_connections 2024-04-25T11:19:27.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.965+0000 7fa003c11640 1 -- 172.21.15.106:0/3719591207 wait complete. 2024-04-25T11:19:27.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 Processor -- start 2024-04-25T11:19:27.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 -- start start 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ffc0b4b00 0x7f9ffc150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc15d6f0 0x7f9ffc1f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc160f10 0x7f9ffc14ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc0d1b30 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc0d19b0 con 0x7f9ffc0b4b00 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc169690 con 0x7f9ffc160f10 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc15d6f0 0x7f9ffc1f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc15d6f0 0x7f9ffc1f7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45960/0 (socket says 172.21.15.106:45960) 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 -- 172.21.15.106:0/4005949502 learned_addr learned my addr 172.21.15.106:0/4005949502 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:27.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0021ae640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc160f10 0x7f9ffc14ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0019ad640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ffc0b4b00 0x7f9ffc150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 -- 172.21.15.106:0/4005949502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc160f10 msgr2=0x7f9ffc14ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc160f10 0x7f9ffc14ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 -- 172.21.15.106:0/4005949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ffc0b4b00 msgr2=0x7f9ffc150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ffc0b4b00 0x7f9ffc150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 -- 172.21.15.106:0/4005949502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc1556e0 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa0011ac640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc15d6f0 0x7f9ffc1f7e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c370 tx=0x7f9fec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7f9ff27fc640 1 -- 172.21.15.106:0/4005949502 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec017070 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7f9ff27fc640 1 -- 172.21.15.106:0/4005949502 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec005cf0 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc1f83a0 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7f9ff27fc640 1 -- 172.21.15.106:0/4005949502 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0072f0 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc155c60 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.969+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc155420 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.973+0000 7f9ff27fc640 1 -- 172.21.15.106:0/4005949502 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9fec01a070 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.973+0000 7f9ff27fc640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9fe006f400 0x7f9fe0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.973+0000 7f9ff27fc640 1 -- 172.21.15.106:0/4005949502 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f9fec08e580 con 0x7f9ffc15d6f0 2024-04-25T11:19:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.973+0000 7fa0019ad640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9fe006f400 0x7f9fe0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.973+0000 7fa0019ad640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9fe006f400 0x7f9fe0071910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80ae000 tx=0x7f9ff8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:27.977+0000 7f9ff27fc640 1 -- 172.21.15.106:0/4005949502 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ffc1556e0 con 0x7f9ffc15d6f0 2024-04-25T11:19:28.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.137+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ffc1572f0 con 0x7f9ffc15d6f0 2024-04-25T11:19:28.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.137+0000 7f9ff27fc640 1 -- 172.21.15.106:0/4005949502 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16654 (secure 0 0 0) 0x7f9fec05b500 con 0x7f9ffc15d6f0 2024-04-25T11:19:28.138 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:28.138 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":19,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:26.806248+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:28.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9fe006f400 msgr2=0x7f9fe0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9fe006f400 0x7f9fe0071910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80ae000 tx=0x7f9ff8008040 comp rx=0 tx=0).stop 2024-04-25T11:19:28.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc15d6f0 msgr2=0x7f9ffc1f7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc15d6f0 0x7f9ffc1f7e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c370 tx=0x7f9fec00c830 comp rx=0 tx=0).stop 2024-04-25T11:19:28.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 shutdown_connections 2024-04-25T11:19:28.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ffc160f10 0x7f9ffc14ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9fe006f400 0x7f9fe0071910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ffc15d6f0 0x7f9ffc1f7e60 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c370 tx=0x7f9fec00c830 comp rx=0 tx=0).stop 2024-04-25T11:19:28.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 --2- 172.21.15.106:0/4005949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ffc0b4b00 0x7f9ffc150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 >> 172.21.15.106:0/4005949502 conn(0x7f9ffc06c7c0 msgr2=0x7f9ffc0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:28.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 shutdown_connections 2024-04-25T11:19:28.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.141+0000 7fa003c11640 1 -- 172.21.15.106:0/4005949502 wait complete. 2024-04-25T11:19:28.159 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:19:28.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f3671766640 1 Processor -- start 2024-04-25T11:19:28.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f3671766640 1 -- start start 2024-04-25T11:19:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f3671766640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f366c0b0b60 0x7f366c15fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f3671766640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c160300 0x7f366c169960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f3671766640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366c0af390 con 0x7f366c0b5730 2024-04-25T11:19:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f3671766640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f366c160c00 con 0x7f366c0b0b60 2024-04-25T11:19:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f3671766640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f366c160ee0 con 0x7f366c160300 2024-04-25T11:19:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f366a7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f366c0b0b60 0x7f366c15fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f366a7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f366c0b0b60 0x7f366c15fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45972/0 (socket says 172.21.15.106:45972) 2024-04-25T11:19:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f366a7fc640 1 -- 172.21.15.106:0/1123602308 learned_addr learned my addr 172.21.15.106:0/1123602308 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:28.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f366b7fe640 1 --2- 172.21.15.106:0/1123602308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c160300 0x7f366c169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f366a7fc640 1 -- 172.21.15.106:0/1123602308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c160300 msgr2=0x7f366c169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:28.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f366a7fc640 1 --2- 172.21.15.106:0/1123602308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c160300 0x7f366c169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f366a7fc640 1 -- 172.21.15.106:0/1123602308 >> v1:172.21.15.134:6789/0 conn(0x7f366c0b5730 legacy=0x7f366c0b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:28.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.301+0000 7f366a7fc640 1 -- 172.21.15.106:0/1123602308 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c1611c0 con 0x7f366c0b0b60 2024-04-25T11:19:28.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f366a7fc640 1 --2- 172.21.15.106:0/1123602308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c0b0b60 0x7f366c15fdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f366c0af560 tx=0x7f365401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24da59ca633a869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3669ffb640 1 -- 172.21.15.106:0/1123602308 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540aa610 con 0x7f366c0b0b60 2024-04-25T11:19:28.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3669ffb640 1 -- 172.21.15.106:0/1123602308 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36540af070 con 0x7f366c0b0b60 2024-04-25T11:19:28.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3669ffb640 1 -- 172.21.15.106:0/1123602308 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540aa9d0 con 0x7f366c0b0b60 2024-04-25T11:19:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 -- 172.21.15.106:0/1123602308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c0b0b60 msgr2=0x7f366c15fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 --2- 172.21.15.106:0/1123602308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c0b0b60 0x7f366c15fdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f366c0af560 tx=0x7f365401c4d0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 -- 172.21.15.106:0/1123602308 shutdown_connections 2024-04-25T11:19:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 --2- 172.21.15.106:0/1123602308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c160300 0x7f366c169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 --2- 172.21.15.106:0/1123602308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c0b0b60 0x7f366c15fdc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 -- 172.21.15.106:0/1123602308 >> 172.21.15.106:0/1123602308 conn(0x7f366c09ba30 msgr2=0x7f366c0b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 -- 172.21.15.106:0/1123602308 shutdown_connections 2024-04-25T11:19:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 -- 172.21.15.106:0/1123602308 wait complete. 2024-04-25T11:19:28.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 Processor -- start 2024-04-25T11:19:28.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.305+0000 7f3671766640 1 -- start start 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c0b0b60 0x7f366c14bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f366c0b5730 0x7f366c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c160300 0x7f366c15ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f366c16ca50 con 0x7f366c160300 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f366c16c8d0 con 0x7f366c0b5730 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f366c16cbd0 con 0x7f366c0b0b60 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c160300 0x7f366c15ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c160300 0x7f366c15ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45986/0 (socket says 172.21.15.106:45986) 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 -- 172.21.15.106:0/3101422339 learned_addr learned my addr 172.21.15.106:0/3101422339 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366a7fc640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f366c0b5730 0x7f366c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366affd640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c0b0b60 0x7f366c14bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 -- 172.21.15.106:0/3101422339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c0b0b60 msgr2=0x7f366c14bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c0b0b60 0x7f366c14bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 -- 172.21.15.106:0/3101422339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f366c0b5730 msgr2=0x7f366c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f366c0b5730 0x7f366c15e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 -- 172.21.15.106:0/3101422339 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c14cac0 con 0x7f366c160300 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f366b7fe640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c160300 0x7f366c15ebc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f365c00d550 tx=0x7f365c00da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f36637fe640 1 -- 172.21.15.106:0/3101422339 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c0125f0 con 0x7f366c160300 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f36637fe640 1 -- 172.21.15.106:0/3101422339 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365c019070 con 0x7f366c160300 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c1577a0 con 0x7f366c160300 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f36637fe640 1 -- 172.21.15.106:0/3101422339 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c0129d0 con 0x7f366c160300 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c157d60 con 0x7f366c160300 2024-04-25T11:19:28.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c050750 con 0x7f366c160300 2024-04-25T11:19:28.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f36637fe640 1 -- 172.21.15.106:0/3101422339 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f365c012b70 con 0x7f366c160300 2024-04-25T11:19:28.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.309+0000 7f36637fe640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f364c06f400 0x7f364c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.313+0000 7f366affd640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f364c06f400 0x7f364c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.313+0000 7f36637fe640 1 -- 172.21.15.106:0/3101422339 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f365c08f6d0 con 0x7f366c160300 2024-04-25T11:19:28.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.313+0000 7f366affd640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f364c06f400 0x7f364c071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f366c0a06e0 tx=0x7f3664007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.317+0000 7f36637fe640 1 -- 172.21.15.106:0/3101422339 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f365c05c5c0 con 0x7f366c160300 2024-04-25T11:19:28.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:28.513+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f366c159330 con 0x7f366c160300 2024-04-25T11:19:29.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.425+0000 7f36637fe640 1 -- 172.21.15.106:0/3101422339 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f365c05bec0 con 0x7f366c160300 2024-04-25T11:19:29.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f364c06f400 msgr2=0x7f364c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f364c06f400 0x7f364c071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f366c0a06e0 tx=0x7f3664007240 comp rx=0 tx=0).stop 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c160300 msgr2=0x7f366c15ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c160300 0x7f366c15ebc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f365c00d550 tx=0x7f365c00da10 comp rx=0 tx=0).stop 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 shutdown_connections 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f364c06f400 0x7f364c071910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c160300 0x7f366c15ebc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f366c0b5730 0x7f366c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 --2- 172.21.15.106:0/3101422339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f366c0b0b60 0x7f366c14bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 >> 172.21.15.106:0/3101422339 conn(0x7f366c09ba30 msgr2=0x7f366c161760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 shutdown_connections 2024-04-25T11:19:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.429+0000 7f3671766640 1 -- 172.21.15.106:0/3101422339 wait complete. 2024-04-25T11:19:29.453 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:19:29.453 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:19:29.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.609+0000 7f64379c6640 1 Processor -- start 2024-04-25T11:19:29.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.609+0000 7f64379c6640 1 -- start start 2024-04-25T11:19:29.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f64379c6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64300cd930 0x7f64300d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f64379c6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64300cdd40 con 0x7f64300d1c80 2024-04-25T11:19:29.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f64379c6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64300d2090 con 0x7f6430143fa0 2024-04-25T11:19:29.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f64379c6640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6430073300 con 0x7f64300cd930 2024-04-25T11:19:29.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f6434f61640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64300cd930 0x7f64300d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f6434f61640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64300cd930 0x7f64300d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46000/0 (socket says 172.21.15.106:46000) 2024-04-25T11:19:29.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f6434f61640 1 -- 172.21.15.106:0/1025792552 learned_addr learned my addr 172.21.15.106:0/1025792552 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:29.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f6435f63640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f64300d1c80 0x7f64300b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56938/0 (socket says 172.21.15.106:56938) 2024-04-25T11:19:29.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f6434f61640 1 -- 172.21.15.106:0/1025792552 >> v1:172.21.15.134:6789/0 conn(0x7f6430143fa0 legacy=0x7f6430072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:29.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f6434f61640 1 -- 172.21.15.106:0/1025792552 >> v1:172.21.15.106:6789/0 conn(0x7f64300d1c80 legacy=0x7f64300b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f6434f61640 1 -- 172.21.15.106:0/1025792552 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64300735e0 con 0x7f64300cd930 2024-04-25T11:19:29.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.613+0000 7f6427fff640 1 -- 172.21.15.106:0/1025792552 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530589509 0 0) 0x7f64280014a0 con 0x7f64300d1c80 2024-04-25T11:19:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f6434f61640 1 --2- 172.21.15.106:0/1025792552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300cd930 0x7f64300d1740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6420009870 tx=0x7f642001c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ecb047b4a39b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f6427fff640 1 -- 172.21.15.106:0/1025792552 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200b0070 con 0x7f64300cd930 2024-04-25T11:19:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f6427fff640 1 -- 172.21.15.106:0/1025792552 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64200ab600 con 0x7f64300cd930 2024-04-25T11:19:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f6427fff640 1 -- 172.21.15.106:0/1025792552 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200ab9d0 con 0x7f64300cd930 2024-04-25T11:19:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 -- 172.21.15.106:0/1025792552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300cd930 msgr2=0x7f64300d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 --2- 172.21.15.106:0/1025792552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300cd930 0x7f64300d1740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6420009870 tx=0x7f642001c320 comp rx=0 tx=0).stop 2024-04-25T11:19:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 -- 172.21.15.106:0/1025792552 shutdown_connections 2024-04-25T11:19:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 --2- 172.21.15.106:0/1025792552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300cd930 0x7f64300d1740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 -- 172.21.15.106:0/1025792552 >> 172.21.15.106:0/1025792552 conn(0x7f643006c9f0 msgr2=0x7f643006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:29.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 -- 172.21.15.106:0/1025792552 shutdown_connections 2024-04-25T11:19:29.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 -- 172.21.15.106:0/1025792552 wait complete. 2024-04-25T11:19:29.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 Processor -- start 2024-04-25T11:19:29.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.617+0000 7f64379c6640 1 -- start start 2024-04-25T11:19:29.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64300cd930 0x7f64300b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300d1c80 0x7f64300b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6430143fa0 0x7f6430157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f643016bb20 con 0x7f64300d1c80 2024-04-25T11:19:29.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f643016b9a0 con 0x7f64300cd930 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f643016bca0 con 0x7f6430143fa0 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300d1c80 0x7f64300b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300d1c80 0x7f64300b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46014/0 (socket says 172.21.15.106:46014) 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 -- 172.21.15.106:0/2631941887 learned_addr learned my addr 172.21.15.106:0/2631941887 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 -- 172.21.15.106:0/2631941887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6430143fa0 msgr2=0x7f6430157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6430143fa0 0x7f6430157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 -- 172.21.15.106:0/2631941887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64300cd930 msgr2=0x7f64300b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6435762640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64300cd930 0x7f64300b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64300cd930 0x7f64300b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 -- 172.21.15.106:0/2631941887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643015f380 con 0x7f64300d1c80 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6434f61640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300d1c80 0x7f64300b35d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6420018f30 tx=0x7f642001ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6425ffb640 1 -- 172.21.15.106:0/2631941887 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200b0040 con 0x7f64300d1c80 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6425ffb640 1 -- 172.21.15.106:0/2631941887 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64200bc070 con 0x7f64300d1c80 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6430160f10 con 0x7f64300d1c80 2024-04-25T11:19:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6425ffb640 1 -- 172.21.15.106:0/2631941887 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200b6860 con 0x7f64300d1c80 2024-04-25T11:19:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643015f0a0 con 0x7f64300d1c80 2024-04-25T11:19:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6430050750 con 0x7f64300d1c80 2024-04-25T11:19:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6425ffb640 1 -- 172.21.15.106:0/2631941887 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f64200b6a00 con 0x7f64300d1c80 2024-04-25T11:19:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6425ffb640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f641406f430 0x7f6414071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6425ffb640 1 -- 172.21.15.106:0/2631941887 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6420129750 con 0x7f64300d1c80 2024-04-25T11:19:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6435762640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f641406f430 0x7f6414071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.621+0000 7f6435762640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f641406f430 0x7f6414071940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f642c00a750 tx=0x7f642c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:29.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.625+0000 7f6425ffb640 1 -- 172.21.15.106:0/2631941887 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64300d2090 con 0x7f64300d1c80 2024-04-25T11:19:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:29.821+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f64300b4e20 con 0x7f64300d1c80 2024-04-25T11:19:30.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.437+0000 7f6425ffb640 1 -- 172.21.15.106:0/2631941887 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f64200f6640 con 0x7f64300d1c80 2024-04-25T11:19:30.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f641406f430 msgr2=0x7f6414071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f641406f430 0x7f6414071940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f642c00a750 tx=0x7f642c008040 comp rx=0 tx=0).stop 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300d1c80 msgr2=0x7f64300b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300d1c80 0x7f64300b35d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6420018f30 tx=0x7f642001ab60 comp rx=0 tx=0).stop 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 shutdown_connections 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6430143fa0 0x7f6430157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f641406f430 0x7f6414071940 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64300d1c80 0x7f64300b35d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 --2- 172.21.15.106:0/2631941887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64300cd930 0x7f64300b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 >> 172.21.15.106:0/2631941887 conn(0x7f643006c9f0 msgr2=0x7f64301462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 shutdown_connections 2024-04-25T11:19:30.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.441+0000 7f64379c6640 1 -- 172.21.15.106:0/2631941887 wait complete. 2024-04-25T11:19:30.461 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:19:30.462 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:30.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb834c9640 1 Processor -- start 2024-04-25T11:19:30.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb834c9640 1 -- start start 2024-04-25T11:19:30.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb834c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c0d1ff0 0x7fdb7c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb834c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdb7c164280 0x7fdb7c164690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb834c9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb7c164bd0 0x7fdb7c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb834c9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7c0d2940 con 0x7fdb7c164bd0 2024-04-25T11:19:30.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb834c9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7c0ce620 con 0x7fdb7c164280 2024-04-25T11:19:30.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb834c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7c0b8ee0 con 0x7fdb7c0d1ff0 2024-04-25T11:19:30.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81a66640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb7c164bd0 0x7fdb7c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81a66640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb7c164bd0 0x7fdb7c0ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46026/0 (socket says 172.21.15.106:46026) 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81a66640 1 -- 172.21.15.106:0/3478953493 learned_addr learned my addr 172.21.15.106:0/3478953493 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81265640 1 --2- 172.21.15.106:0/3478953493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c0d1ff0 0x7fdb7c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb80a64640 1 --2- 172.21.15.106:0/3478953493 >> v2:172.21.15.134:3300/0 conn(0x7fdb7c164280 0x7fdb7c164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81a66640 1 -- 172.21.15.106:0/3478953493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c0d1ff0 msgr2=0x7fdb7c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81a66640 1 --2- 172.21.15.106:0/3478953493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c0d1ff0 0x7fdb7c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81a66640 1 -- 172.21.15.106:0/3478953493 >> v2:172.21.15.134:3300/0 conn(0x7fdb7c164280 msgr2=0x7fdb7c164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81a66640 1 --2- 172.21.15.106:0/3478953493 >> v2:172.21.15.134:3300/0 conn(0x7fdb7c164280 0x7fdb7c164690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb81a66640 1 -- 172.21.15.106:0/3478953493 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c0b8c40 con 0x7fdb7c164bd0 2024-04-25T11:19:30.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.613+0000 7fdb80a64640 1 --2- 172.21.15.106:0/3478953493 >> v2:172.21.15.134:3300/0 conn(0x7fdb7c164280 0x7fdb7c164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:30.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb81a66640 1 --2- 172.21.15.106:0/3478953493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164bd0 0x7fdb7c0ceab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c073f80 tx=0x7fdb6c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c7e87e75c24b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb7bfff640 1 -- 172.21.15.106:0/3478953493 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0aa610 con 0x7fdb7c164bd0 2024-04-25T11:19:30.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb7bfff640 1 -- 172.21.15.106:0/3478953493 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6c0af070 con 0x7fdb7c164bd0 2024-04-25T11:19:30.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb7bfff640 1 -- 172.21.15.106:0/3478953493 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0aa9d0 con 0x7fdb7c164bd0 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 -- 172.21.15.106:0/3478953493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164bd0 msgr2=0x7fdb7c0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 --2- 172.21.15.106:0/3478953493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164bd0 0x7fdb7c0ceab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c073f80 tx=0x7fdb6c01c4f0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 -- 172.21.15.106:0/3478953493 shutdown_connections 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 --2- 172.21.15.106:0/3478953493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164bd0 0x7fdb7c0ceab0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 --2- 172.21.15.106:0/3478953493 >> v2:172.21.15.134:3300/0 conn(0x7fdb7c164280 0x7fdb7c164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 --2- 172.21.15.106:0/3478953493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c0d1ff0 0x7fdb7c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 -- 172.21.15.106:0/3478953493 >> 172.21.15.106:0/3478953493 conn(0x7fdb7c0d0040 msgr2=0x7fdb7c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 -- 172.21.15.106:0/3478953493 shutdown_connections 2024-04-25T11:19:30.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.617+0000 7fdb834c9640 1 -- 172.21.15.106:0/3478953493 wait complete. 2024-04-25T11:19:30.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 Processor -- start 2024-04-25T11:19:30.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 -- start start 2024-04-25T11:19:30.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0d1ff0 0x7fdb7c1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164280 0x7fdb7c154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c164bd0 0x7fdb7c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c0d16a0 con 0x7fdb7c164280 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c0d1520 con 0x7fdb7c0d1ff0 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7c0cf800 con 0x7fdb7c164bd0 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164280 0x7fdb7c154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164280 0x7fdb7c154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46040/0 (socket says 172.21.15.106:46040) 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 -- 172.21.15.106:0/4203441515 learned_addr learned my addr 172.21.15.106:0/4203441515 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb81265640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0d1ff0 0x7fdb7c1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb81a66640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c164bd0 0x7fdb7c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 -- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c164bd0 msgr2=0x7fdb7c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c164bd0 0x7fdb7c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 -- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0d1ff0 msgr2=0x7fdb7c1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0d1ff0 0x7fdb7c1533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 -- 172.21.15.106:0/4203441515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c1525e0 con 0x7fdb7c164280 2024-04-25T11:19:30.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb81265640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0d1ff0 0x7fdb7c1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:30.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb81a66640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c164bd0 0x7fdb7c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:30.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb80a64640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164280 0x7fdb7c154540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c820 tx=0x7fdb7000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb79ffb640 1 -- 172.21.15.106:0/4203441515 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70015070 con 0x7fdb7c164280 2024-04-25T11:19:30.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb79ffb640 1 -- 172.21.15.106:0/4203441515 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb700079d0 con 0x7fdb7c164280 2024-04-25T11:19:30.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c1528c0 con 0x7fdb7c164280 2024-04-25T11:19:30.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb79ffb640 1 -- 172.21.15.106:0/4203441515 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70007da0 con 0x7fdb7c164280 2024-04-25T11:19:30.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.621+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c161f90 con 0x7fdb7c164280 2024-04-25T11:19:30.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.625+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7c0d2720 con 0x7fdb7c164280 2024-04-25T11:19:30.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.625+0000 7fdb79ffb640 1 -- 172.21.15.106:0/4203441515 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb70018070 con 0x7fdb7c164280 2024-04-25T11:19:30.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.625+0000 7fdb79ffb640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb6806f430 0x7fdb68071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.625+0000 7fdb79ffb640 1 -- 172.21.15.106:0/4203441515 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdb7008f270 con 0x7fdb7c164280 2024-04-25T11:19:30.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.625+0000 7fdb81265640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb6806f430 0x7fdb68071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.625+0000 7fdb81265640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb6806f430 0x7fdb68071940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb74007da0 tx=0x7fdb740072f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.629+0000 7fdb79ffb640 1 -- 172.21.15.106:0/4203441515 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb7005c0c0 con 0x7fdb7c164280 2024-04-25T11:19:30.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.809+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb7c0b90f0 con 0x7fdb7c164280 2024-04-25T11:19:30.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb79ffb640 1 -- 172.21.15.106:0/4203441515 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fdb7005b9c0 con 0x7fdb7c164280 2024-04-25T11:19:30.811 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:30.811 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:26.806158+0000","modified":"2024-04-25T11:19:29.823480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:19:30.811 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb6806f430 msgr2=0x7fdb68071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb6806f430 0x7fdb68071940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb74007da0 tx=0x7fdb740072f0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164280 msgr2=0x7fdb7c154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164280 0x7fdb7c154540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c820 tx=0x7fdb7000cce0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 shutdown_connections 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7c164bd0 0x7fdb7c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb6806f430 0x7fdb68071940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c164280 0x7fdb7c154540 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 --2- 172.21.15.106:0/4203441515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0d1ff0 0x7fdb7c1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 >> 172.21.15.106:0/4203441515 conn(0x7fdb7c0d0040 msgr2=0x7fdb7c163810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 shutdown_connections 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.813+0000 7fdb834c9640 1 -- 172.21.15.106:0/4203441515 wait complete. 2024-04-25T11:19:30.830 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:30.939 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-04-25T11:19:30.938+0000 7f40a076d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:19:30.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a505ce640 1 Processor -- start 2024-04-25T11:19:30.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a505ce640 1 -- start start 2024-04-25T11:19:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a505ce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a4815e0a0 0x7f5a48073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a505ce640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5a480d0ce0 0x7f5a480d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a505ce640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a480ce050 con 0x7f5a480ce850 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a505ce640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4815e6b0 con 0x7f5a480d0ce0 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a505ce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4815e990 con 0x7f5a4815e0a0 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4db69640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5a480ce850 0x7f5a480cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56954/0 (socket says 172.21.15.106:56954) 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4db69640 1 -- 172.21.15.106:0/154366869 learned_addr learned my addr 172.21.15.106:0/154366869 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162637374 0 0) 0x7f5a480ce050 con 0x7f5a480ce850 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4e36a640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a4815e0a0 0x7f5a48073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4eb6b640 1 --2- 172.21.15.106:0/154366869 >> v2:172.21.15.134:3300/0 conn(0x7f5a480d0ce0 0x7f5a480d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a34003530 con 0x7f5a480ce850 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1972548163 0 0) 0x7f5a34003530 con 0x7f5a480ce850 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a4815e0a0 msgr2=0x7f5a48073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a4815e0a0 0x7f5a48073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 >> v2:172.21.15.134:3300/0 conn(0x7f5a480d0ce0 msgr2=0x7f5a480d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 --2- 172.21.15.106:0/154366869 >> v2:172.21.15.134:3300/0 conn(0x7f5a480d0ce0 0x7f5a480d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4815ec70 con 0x7f5a480ce850 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f5a3c001680 con 0x7f5a480ce850 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 >> v1:172.21.15.106:6789/0 conn(0x7f5a480ce850 legacy=0x7f5a480cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a340039c0 0x7f5a34003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a4815e0a0 0x7f5a34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a34005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4815e6b0 con 0x7f5a34004d50 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4815e990 con 0x7f5a340039c0 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3400b9d0 con 0x7f5a4815e0a0 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4eb6b640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a34005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4e36a640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a340039c0 0x7f5a34003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4db69640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a4815e0a0 0x7f5a34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4eb6b640 1 -- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a4815e0a0 msgr2=0x7f5a34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.977+0000 7f5a4eb6b640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a4815e0a0 0x7f5a34004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a4eb6b640 1 -- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a340039c0 msgr2=0x7f5a34003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a4eb6b640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a340039c0 0x7f5a34003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a4eb6b640 1 -- 172.21.15.106:0/154366869 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a34003530 con 0x7f5a34004d50 2024-04-25T11:19:30.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a4eb6b640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a34005160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009fc0 tx=0x7f5a4001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2942df37267317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a400b2070 con 0x7f5a34004d50 2024-04-25T11:19:30.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a400a5c90 con 0x7f5a34004d50 2024-04-25T11:19:30.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a4d368640 1 -- 172.21.15.106:0/154366869 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40009c10 con 0x7f5a34004d50 2024-04-25T11:19:30.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a505ce640 1 -- 172.21.15.106:0/154366869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 msgr2=0x7f5a34005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a505ce640 1 --2- 172.21.15.106:0/154366869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a34005160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009fc0 tx=0x7f5a4001cc80 comp rx=0 tx=0).stop 2024-04-25T11:19:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a4eb6b640 1 -- 172.21.15.106:0/154366869 reap_dead start 2024-04-25T11:19:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a505ce640 1 -- 172.21.15.106:0/154366869 shutdown_connections 2024-04-25T11:19:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a505ce640 1 -- 172.21.15.106:0/154366869 >> 172.21.15.106:0/154366869 conn(0x7f5a4806c7c0 msgr2=0x7f5a4806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a505ce640 1 -- 172.21.15.106:0/154366869 shutdown_connections 2024-04-25T11:19:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.981+0000 7f5a505ce640 1 -- 172.21.15.106:0/154366869 wait complete. 2024-04-25T11:19:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a505ce640 1 Processor -- start 2024-04-25T11:19:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a505ce640 1 -- start start 2024-04-25T11:19:30.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a505ce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a480abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a505ce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a480aa760 0x7f5a480aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a505ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a480ab0b0 0x7f5a48131a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a505ce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5a480d0860 con 0x7f5a34004d50 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a505ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a480d06e0 con 0x7f5a480ab0b0 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a505ce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4815f270 con 0x7f5a480aa760 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a480abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a480abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46062/0 (socket says 172.21.15.106:46062) 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 -- 172.21.15.106:0/495918973 learned_addr learned my addr 172.21.15.106:0/495918973 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4eb6b640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a480ab0b0 0x7f5a48131a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4db69640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a480aa760 0x7f5a480aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 -- 172.21.15.106:0/495918973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a480aa760 msgr2=0x7f5a480aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a480aa760 0x7f5a480aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 -- 172.21.15.106:0/495918973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a480ab0b0 msgr2=0x7f5a48131a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a480ab0b0 0x7f5a48131a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 -- 172.21.15.106:0/495918973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a480afd10 con 0x7f5a34004d50 2024-04-25T11:19:30.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a4e36a640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a480abb70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a44011590 tx=0x7f5a44011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a3affd640 1 -- 172.21.15.106:0/495918973 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44015930 con 0x7f5a34004d50 2024-04-25T11:19:30.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a3affd640 1 -- 172.21.15.106:0/495918973 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a44015ad0 con 0x7f5a34004d50 2024-04-25T11:19:30.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.985+0000 7f5a3affd640 1 -- 172.21.15.106:0/495918973 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4400f460 con 0x7f5a34004d50 2024-04-25T11:19:30.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.989+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a480b0150 con 0x7f5a34004d50 2024-04-25T11:19:30.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.989+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a48131220 con 0x7f5a34004d50 2024-04-25T11:19:30.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.989+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a48141340 con 0x7f5a34004d50 2024-04-25T11:19:30.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.989+0000 7f5a3affd640 1 -- 172.21.15.106:0/495918973 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5a4400f600 con 0x7f5a34004d50 2024-04-25T11:19:30.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.989+0000 7f5a3affd640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5a2c06f400 0x7f5a2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.989+0000 7f5a4db69640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5a2c06f400 0x7f5a2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.989+0000 7f5a3affd640 1 -- 172.21.15.106:0/495918973 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5a4408fbc0 con 0x7f5a34004d50 2024-04-25T11:19:30.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.989+0000 7f5a4db69640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5a2c06f400 0x7f5a2c071910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000f70 tx=0x7f5a3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:30.993+0000 7f5a3affd640 1 -- 172.21.15.106:0/495918973 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a44019030 con 0x7f5a34004d50 2024-04-25T11:19:31.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.149+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a48073400 con 0x7f5a34004d50 2024-04-25T11:19:31.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.149+0000 7f5a3affd640 1 -- 172.21.15.106:0/495918973 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f5a4405cab0 con 0x7f5a34004d50 2024-04-25T11:19:31.148 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:31.148 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:31.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.149+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5a2c06f400 msgr2=0x7f5a2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.149+0000 7f5a505ce640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5a2c06f400 0x7f5a2c071910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000f70 tx=0x7f5a3c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 msgr2=0x7f5a480abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a480abb70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a44011590 tx=0x7f5a44011a50 comp rx=0 tx=0).stop 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 shutdown_connections 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a480ab0b0 0x7f5a48131a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a480aa760 0x7f5a480aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5a2c06f400 0x7f5a2c071910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 --2- 172.21.15.106:0/495918973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a34004d50 0x7f5a480abb70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 >> 172.21.15.106:0/495918973 conn(0x7f5a4806c7c0 msgr2=0x7f5a4806f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 shutdown_connections 2024-04-25T11:19:31.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.153+0000 7f5a505ce640 1 -- 172.21.15.106:0/495918973 wait complete. 2024-04-25T11:19:31.168 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:19:31.168 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:19:31.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7bfdd640 1 Processor -- start 2024-04-25T11:19:31.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7bfdd640 1 -- start start 2024-04-25T11:19:31.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7bfdd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d741611a0 0x7f6d7415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7bfdd640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d740b0be0 con 0x7f6d740ce580 2024-04-25T11:19:31.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7bfdd640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d740b0ec0 con 0x7f6d74160d90 2024-04-25T11:19:31.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7bfdd640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6d740cecf0 con 0x7f6d741611a0 2024-04-25T11:19:31.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7a57a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d741611a0 0x7f6d7415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d79d79640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6d740ce580 0x7f6d74160680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56956/0 (socket says 172.21.15.106:56956) 2024-04-25T11:19:31.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d79d79640 1 -- 172.21.15.106:0/3613656355 learned_addr learned my addr 172.21.15.106:0/3613656355 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:31.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7a57a640 1 --2- 172.21.15.106:0/3613656355 >> v2:172.21.15.106:3300/0 conn(0x7f6d741611a0 0x7f6d7415d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46064/0 (socket says 172.21.15.106:46064) 2024-04-25T11:19:31.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7a57a640 1 -- 172.21.15.106:0/3613656355 >> v1:172.21.15.134:6789/0 conn(0x7f6d74160d90 legacy=0x7f6d7415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7a57a640 1 -- 172.21.15.106:0/3613656355 >> v1:172.21.15.106:6789/0 conn(0x7f6d740ce580 legacy=0x7f6d74160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.305+0000 7f6d7a57a640 1 -- 172.21.15.106:0/3613656355 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d740b11a0 con 0x7f6d741611a0 2024-04-25T11:19:31.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d7a57a640 1 --2- 172.21.15.106:0/3613656355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d7415d2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009d90 tx=0x7f6d6c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f723939aa622f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d78d77640 1 -- 172.21.15.106:0/3613656355 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0b2070 con 0x7f6d741611a0 2024-04-25T11:19:31.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d78d77640 1 -- 172.21.15.106:0/3613656355 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d6c0a5950 con 0x7f6d741611a0 2024-04-25T11:19:31.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d78d77640 1 -- 172.21.15.106:0/3613656355 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0a5d20 con 0x7f6d741611a0 2024-04-25T11:19:31.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/3613656355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 msgr2=0x7f6d7415d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d7bfdd640 1 --2- 172.21.15.106:0/3613656355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d7415d2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009d90 tx=0x7f6d6c01c870 comp rx=0 tx=0).stop 2024-04-25T11:19:31.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/3613656355 shutdown_connections 2024-04-25T11:19:31.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d7bfdd640 1 --2- 172.21.15.106:0/3613656355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d7415d2c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/3613656355 >> 172.21.15.106:0/3613656355 conn(0x7f6d740b78b0 msgr2=0x7f6d740b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/3613656355 shutdown_connections 2024-04-25T11:19:31.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.309+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/3613656355 wait complete. 2024-04-25T11:19:31.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 Processor -- start 2024-04-25T11:19:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 -- start start 2024-04-25T11:19:31.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d740ce580 0x7f6d7413d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d74160d90 0x7f6d74151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d74152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7416adc0 con 0x7f6d741611a0 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7416ac40 con 0x7f6d74160d90 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6d7416af40 con 0x7f6d740ce580 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d74152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d74152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46080/0 (socket says 172.21.15.106:46080) 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 -- 172.21.15.106:0/2480533465 learned_addr learned my addr 172.21.15.106:0/2480533465 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d79d79640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d740ce580 0x7f6d7413d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d79578640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d74160d90 0x7f6d74151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 -- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d740ce580 msgr2=0x7f6d7413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d740ce580 0x7f6d7413d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 -- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d74160d90 msgr2=0x7f6d74151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d74160d90 0x7f6d74151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 -- 172.21.15.106:0/2480533465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7413e4b0 con 0x7f6d741611a0 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d79d79640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d740ce580 0x7f6d7413d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7a57a640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d74152160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c01a4a0 tx=0x7f6d6c019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d6a7fc640 1 -- 172.21.15.106:0/2480533465 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0b2050 con 0x7f6d741611a0 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d6a7fc640 1 -- 172.21.15.106:0/2480533465 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d6c0be070 con 0x7f6d741611a0 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7413def0 con 0x7f6d741611a0 2024-04-25T11:19:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d6a7fc640 1 -- 172.21.15.106:0/2480533465 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0ada50 con 0x7f6d741611a0 2024-04-25T11:19:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7413b620 con 0x7f6d741611a0 2024-04-25T11:19:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.313+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d740744a0 con 0x7f6d741611a0 2024-04-25T11:19:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.317+0000 7f6d6a7fc640 1 -- 172.21.15.106:0/2480533465 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6d6c0adbf0 con 0x7f6d741611a0 2024-04-25T11:19:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.317+0000 7f6d6a7fc640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6d5806f430 0x7f6d58071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.317+0000 7f6d79d79640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6d5806f430 0x7f6d58071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.317+0000 7f6d6a7fc640 1 -- 172.21.15.106:0/2480533465 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6d6c12ab70 con 0x7f6d741611a0 2024-04-25T11:19:31.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.317+0000 7f6d79d79640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6d5806f430 0x7f6d58071940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6d70007fd0 tx=0x7f6d70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.321+0000 7f6d6a7fc640 1 -- 172.21.15.106:0/2480533465 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d6c016520 con 0x7f6d741611a0 2024-04-25T11:19:31.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.465+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d741609e0 con 0x7f6d5806f430 2024-04-25T11:19:31.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.477+0000 7f6d6a7fc640 1 -- 172.21.15.106:0/2480533465 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251896 (secure 0 0 0) 0x7f6d741609e0 con 0x7f6d5806f430 2024-04-25T11:19:31.480 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:31.482 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:19:31.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6d5806f430 msgr2=0x7f6d58071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6d5806f430 0x7f6d58071940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6d70007fd0 tx=0x7f6d70009040 comp rx=0 tx=0).stop 2024-04-25T11:19:31.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 msgr2=0x7f6d74152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d74152160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c01a4a0 tx=0x7f6d6c019610 comp rx=0 tx=0).stop 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 shutdown_connections 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6d5806f430 0x7f6d58071940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d741611a0 0x7f6d74152160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d74160d90 0x7f6d74151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 --2- 172.21.15.106:0/2480533465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d740ce580 0x7f6d7413d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 >> 172.21.15.106:0/2480533465 conn(0x7f6d740b78b0 msgr2=0x7f6d7415cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 shutdown_connections 2024-04-25T11:19:31.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.481+0000 7f6d7bfdd640 1 -- 172.21.15.106:0/2480533465 wait complete. 2024-04-25T11:19:31.499 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T11:19:30.091646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218968,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755752,"statfs":{"total":773094113280,"available":772869890048,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.705814"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452894+0000","last_change":"2024-04-25T11:19:27.448704+0000","last_active":"2024-04-25T11:19:27.452894+0000","last_peered":"2024-04-25T11:19:27.452894+0000","last_clean":"2024-04-25T11:19:27.452894+0000","last_became_active":"2024-04-25T11:19:27.447782+0000","last_became_peered":"2024-04-25T11:19:27.447782+0000","last_unstale":"2024-04-25T11:19:27.452894+0000","last_undegraded":"2024-04-25T11:19:27.452894+0000","last_fullsized":"2024-04-25T11:19:27.452894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.850268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422162+0000","last_change":"2024-04-25T11:19:27.424499+0000","last_active":"2024-04-25T11:19:28.422162+0000","last_peered":"2024-04-25T11:19:28.422162+0000","last_clean":"2024-04-25T11:19:28.422162+0000","last_became_active":"2024-04-25T11:19:27.424122+0000","last_became_peered":"2024-04-25T11:19:27.424122+0000","last_unstale":"2024-04-25T11:19:28.422162+0000","last_undegraded":"2024-04-25T11:19:28.422162+0000","last_fullsized":"2024-04-25T11:19:28.422162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.429833+0000","last_change":"2024-04-25T11:19:27.422128+0000","last_active":"2024-04-25T11:19:27.429833+0000","last_peered":"2024-04-25T11:19:27.429833+0000","last_clean":"2024-04-25T11:19:27.429833+0000","last_became_active":"2024-04-25T11:19:27.421683+0000","last_became_peered":"2024-04-25T11:19:27.421683+0000","last_unstale":"2024-04-25T11:19:27.429833+0000","last_undegraded":"2024-04-25T11:19:27.429833+0000","last_fullsized":"2024-04-25T11:19:27.429833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:30.664977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818544+0000","last_change":"2024-04-25T11:19:27.439794+0000","last_active":"2024-04-25T11:19:28.818544+0000","last_peered":"2024-04-25T11:19:28.818544+0000","last_clean":"2024-04-25T11:19:28.818544+0000","last_became_active":"2024-04-25T11:19:27.439092+0000","last_became_peered":"2024-04-25T11:19:27.439092+0000","last_unstale":"2024-04-25T11:19:28.818544+0000","last_undegraded":"2024-04-25T11:19:28.818544+0000","last_fullsized":"2024-04-25T11:19:28.818544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:52.757499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452464+0000","last_change":"2024-04-25T11:19:27.447315+0000","last_active":"2024-04-25T11:19:27.452464+0000","last_peered":"2024-04-25T11:19:27.452464+0000","last_clean":"2024-04-25T11:19:27.452464+0000","last_became_active":"2024-04-25T11:19:27.446274+0000","last_became_peered":"2024-04-25T11:19:27.446274+0000","last_unstale":"2024-04-25T11:19:27.452464+0000","last_undegraded":"2024-04-25T11:19:27.452464+0000","last_fullsized":"2024-04-25T11:19:27.452464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817544+0000","last_change":"2024-04-25T11:19:27.446758+0000","last_active":"2024-04-25T11:19:28.817544+0000","last_peered":"2024-04-25T11:19:28.817544+0000","last_clean":"2024-04-25T11:19:28.817544+0000","last_became_active":"2024-04-25T11:19:27.440252+0000","last_became_peered":"2024-04-25T11:19:27.440252+0000","last_unstale":"2024-04-25T11:19:28.817544+0000","last_undegraded":"2024-04-25T11:19:28.817544+0000","last_fullsized":"2024-04-25T11:19:28.817544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:08.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451383+0000","last_change":"2024-04-25T11:19:27.442783+0000","last_active":"2024-04-25T11:19:27.451383+0000","last_peered":"2024-04-25T11:19:27.451383+0000","last_clean":"2024-04-25T11:19:27.451383+0000","last_became_active":"2024-04-25T11:19:27.440913+0000","last_became_peered":"2024-04-25T11:19:27.440913+0000","last_unstale":"2024-04-25T11:19:27.451383+0000","last_undegraded":"2024-04-25T11:19:27.451383+0000","last_fullsized":"2024-04-25T11:19:27.451383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.443767+0000","last_change":"2024-04-25T11:19:27.441574+0000","last_active":"2024-04-25T11:19:27.443767+0000","last_peered":"2024-04-25T11:19:27.443767+0000","last_clean":"2024-04-25T11:19:27.443767+0000","last_became_active":"2024-04-25T11:19:27.441265+0000","last_became_peered":"2024-04-25T11:19:27.441265+0000","last_unstale":"2024-04-25T11:19:27.443767+0000","last_undegraded":"2024-04-25T11:19:27.443767+0000","last_fullsized":"2024-04-25T11:19:27.443767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:04.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.464027+0000","last_change":"2024-04-25T11:19:27.459170+0000","last_active":"2024-04-25T11:19:27.464027+0000","last_peered":"2024-04-25T11:19:27.464027+0000","last_clean":"2024-04-25T11:19:27.464027+0000","last_became_active":"2024-04-25T11:19:27.458755+0000","last_became_peered":"2024-04-25T11:19:27.458755+0000","last_unstale":"2024-04-25T11:19:27.464027+0000","last_undegraded":"2024-04-25T11:19:27.464027+0000","last_fullsized":"2024-04-25T11:19:27.464027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:08.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433030+0000","last_change":"2024-04-25T11:19:27.456078+0000","last_active":"2024-04-25T11:19:28.433030+0000","last_peered":"2024-04-25T11:19:28.433030+0000","last_clean":"2024-04-25T11:19:28.433030+0000","last_became_active":"2024-04-25T11:19:27.455302+0000","last_became_peered":"2024-04-25T11:19:27.455302+0000","last_unstale":"2024-04-25T11:19:28.433030+0000","last_undegraded":"2024-04-25T11:19:28.433030+0000","last_fullsized":"2024-04-25T11:19:28.433030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432803+0000","last_change":"2024-04-25T11:19:27.459132+0000","last_active":"2024-04-25T11:19:28.432803+0000","last_peered":"2024-04-25T11:19:28.432803+0000","last_clean":"2024-04-25T11:19:28.432803+0000","last_became_active":"2024-04-25T11:19:27.458820+0000","last_became_peered":"2024-04-25T11:19:27.458820+0000","last_unstale":"2024-04-25T11:19:28.432803+0000","last_undegraded":"2024-04-25T11:19:28.432803+0000","last_fullsized":"2024-04-25T11:19:28.432803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.067341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442624+0000","last_change":"2024-04-25T11:19:27.439470+0000","last_active":"2024-04-25T11:19:27.442624+0000","last_peered":"2024-04-25T11:19:27.442624+0000","last_clean":"2024-04-25T11:19:27.442624+0000","last_became_active":"2024-04-25T11:19:27.439141+0000","last_became_peered":"2024-04-25T11:19:27.439141+0000","last_unstale":"2024-04-25T11:19:27.442624+0000","last_undegraded":"2024-04-25T11:19:27.442624+0000","last_fullsized":"2024-04-25T11:19:27.442624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:30.664977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.441977+0000","last_change":"2024-04-25T11:19:27.433821+0000","last_active":"2024-04-25T11:19:27.441977+0000","last_peered":"2024-04-25T11:19:27.441977+0000","last_clean":"2024-04-25T11:19:27.441977+0000","last_became_active":"2024-04-25T11:19:27.433427+0000","last_became_peered":"2024-04-25T11:19:27.433427+0000","last_unstale":"2024-04-25T11:19:27.441977+0000","last_undegraded":"2024-04-25T11:19:27.441977+0000","last_fullsized":"2024-04-25T11:19:27.441977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:25.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.434067+0000","last_change":"2024-04-25T11:19:27.428346+0000","last_active":"2024-04-25T11:19:27.434067+0000","last_peered":"2024-04-25T11:19:27.434067+0000","last_clean":"2024-04-25T11:19:27.434067+0000","last_became_active":"2024-04-25T11:19:27.428006+0000","last_became_peered":"2024-04-25T11:19:27.428006+0000","last_unstale":"2024-04-25T11:19:27.434067+0000","last_undegraded":"2024-04-25T11:19:27.434067+0000","last_fullsized":"2024-04-25T11:19:27.434067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421393+0000","last_change":"2024-04-25T11:19:27.428566+0000","last_active":"2024-04-25T11:19:28.421393+0000","last_peered":"2024-04-25T11:19:28.421393+0000","last_clean":"2024-04-25T11:19:28.421393+0000","last_became_active":"2024-04-25T11:19:27.427282+0000","last_became_peered":"2024-04-25T11:19:27.427282+0000","last_unstale":"2024-04-25T11:19:28.421393+0000","last_undegraded":"2024-04-25T11:19:28.421393+0000","last_fullsized":"2024-04-25T11:19:28.421393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817322+0000","last_change":"2024-04-25T11:19:27.448380+0000","last_active":"2024-04-25T11:19:28.817322+0000","last_peered":"2024-04-25T11:19:28.817322+0000","last_clean":"2024-04-25T11:19:28.817322+0000","last_became_active":"2024-04-25T11:19:27.447971+0000","last_became_peered":"2024-04-25T11:19:27.447971+0000","last_unstale":"2024-04-25T11:19:28.817322+0000","last_undegraded":"2024-04-25T11:19:28.817322+0000","last_fullsized":"2024-04-25T11:19:28.817322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.850268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818713+0000","last_change":"2024-04-25T11:19:27.452389+0000","last_active":"2024-04-25T11:19:28.818713+0000","last_peered":"2024-04-25T11:19:28.818713+0000","last_clean":"2024-04-25T11:19:28.818713+0000","last_became_active":"2024-04-25T11:19:27.451910+0000","last_became_peered":"2024-04-25T11:19:27.451910+0000","last_unstale":"2024-04-25T11:19:28.818713+0000","last_undegraded":"2024-04-25T11:19:28.818713+0000","last_fullsized":"2024-04-25T11:19:28.818713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424567+0000","last_change":"2024-04-25T11:19:27.437476+0000","last_active":"2024-04-25T11:19:28.424567+0000","last_peered":"2024-04-25T11:19:28.424567+0000","last_clean":"2024-04-25T11:19:28.424567+0000","last_became_active":"2024-04-25T11:19:27.436712+0000","last_became_peered":"2024-04-25T11:19:27.436712+0000","last_unstale":"2024-04-25T11:19:28.424567+0000","last_undegraded":"2024-04-25T11:19:28.424567+0000","last_fullsized":"2024-04-25T11:19:28.424567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452583+0000","last_change":"2024-04-25T11:19:27.445559+0000","last_active":"2024-04-25T11:19:27.452583+0000","last_peered":"2024-04-25T11:19:27.452583+0000","last_clean":"2024-04-25T11:19:27.452583+0000","last_became_active":"2024-04-25T11:19:27.445058+0000","last_became_peered":"2024-04-25T11:19:27.445058+0000","last_unstale":"2024-04-25T11:19:27.452583+0000","last_undegraded":"2024-04-25T11:19:27.452583+0000","last_fullsized":"2024-04-25T11:19:27.452583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432846+0000","last_change":"2024-04-25T11:19:27.441427+0000","last_active":"2024-04-25T11:19:28.432846+0000","last_peered":"2024-04-25T11:19:28.432846+0000","last_clean":"2024-04-25T11:19:28.432846+0000","last_became_active":"2024-04-25T11:19:27.441087+0000","last_became_peered":"2024-04-25T11:19:27.441087+0000","last_unstale":"2024-04-25T11:19:28.432846+0000","last_undegraded":"2024-04-25T11:19:28.432846+0000","last_fullsized":"2024-04-25T11:19:28.432846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451023+0000","last_change":"2024-04-25T11:19:27.445781+0000","last_active":"2024-04-25T11:19:27.451023+0000","last_peered":"2024-04-25T11:19:27.451023+0000","last_clean":"2024-04-25T11:19:27.451023+0000","last_became_active":"2024-04-25T11:19:27.445240+0000","last_became_peered":"2024-04-25T11:19:27.445240+0000","last_unstale":"2024-04-25T11:19:27.451023+0000","last_undegraded":"2024-04-25T11:19:27.451023+0000","last_fullsized":"2024-04-25T11:19:27.451023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:13.878942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817748+0000","last_change":"2024-04-25T11:19:27.453395+0000","last_active":"2024-04-25T11:19:28.817748+0000","last_peered":"2024-04-25T11:19:28.817748+0000","last_clean":"2024-04-25T11:19:28.817748+0000","last_became_active":"2024-04-25T11:19:27.452942+0000","last_became_peered":"2024-04-25T11:19:27.452942+0000","last_unstale":"2024-04-25T11:19:28.817748+0000","last_undegraded":"2024-04-25T11:19:28.817748+0000","last_fullsized":"2024-04-25T11:19:28.817748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:39.163529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.434048+0000","last_change":"2024-04-25T11:19:27.427559+0000","last_active":"2024-04-25T11:19:27.434048+0000","last_peered":"2024-04-25T11:19:27.434048+0000","last_clean":"2024-04-25T11:19:27.434048+0000","last_became_active":"2024-04-25T11:19:27.421099+0000","last_became_peered":"2024-04-25T11:19:27.421099+0000","last_unstale":"2024-04-25T11:19:27.434048+0000","last_undegraded":"2024-04-25T11:19:27.434048+0000","last_fullsized":"2024-04-25T11:19:27.434048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.963336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817459+0000","last_change":"2024-04-25T11:19:27.440162+0000","last_active":"2024-04-25T11:19:28.817459+0000","last_peered":"2024-04-25T11:19:28.817459+0000","last_clean":"2024-04-25T11:19:28.817459+0000","last_became_active":"2024-04-25T11:19:27.439384+0000","last_became_peered":"2024-04-25T11:19:27.439384+0000","last_unstale":"2024-04-25T11:19:28.817459+0000","last_undegraded":"2024-04-25T11:19:28.817459+0000","last_fullsized":"2024-04-25T11:19:28.817459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.398422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442680+0000","last_change":"2024-04-25T11:19:27.438911+0000","last_active":"2024-04-25T11:19:27.442680+0000","last_peered":"2024-04-25T11:19:27.442680+0000","last_clean":"2024-04-25T11:19:27.442680+0000","last_became_active":"2024-04-25T11:19:27.437950+0000","last_became_peered":"2024-04-25T11:19:27.437950+0000","last_unstale":"2024-04-25T11:19:27.442680+0000","last_undegraded":"2024-04-25T11:19:27.442680+0000","last_fullsized":"2024-04-25T11:19:27.442680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:13.878942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433178+0000","last_change":"2024-04-25T11:19:27.447613+0000","last_active":"2024-04-25T11:19:28.433178+0000","last_peered":"2024-04-25T11:19:28.433178+0000","last_clean":"2024-04-25T11:19:28.433178+0000","last_became_active":"2024-04-25T11:19:27.447314+0000","last_became_peered":"2024-04-25T11:19:27.447314+0000","last_unstale":"2024-04-25T11:19:28.433178+0000","last_undegraded":"2024-04-25T11:19:28.433178+0000","last_fullsized":"2024-04-25T11:19:28.433178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421840+0000","last_change":"2024-04-25T11:19:27.436767+0000","last_active":"2024-04-25T11:19:28.421840+0000","last_peered":"2024-04-25T11:19:28.421840+0000","last_clean":"2024-04-25T11:19:28.421840+0000","last_became_active":"2024-04-25T11:19:27.436392+0000","last_became_peered":"2024-04-25T11:19:27.436392+0000","last_unstale":"2024-04-25T11:19:28.421840+0000","last_undegraded":"2024-04-25T11:19:28.421840+0000","last_fullsized":"2024-04-25T11:19:28.421840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442607+0000","last_change":"2024-04-25T11:19:27.438536+0000","last_active":"2024-04-25T11:19:27.442607+0000","last_peered":"2024-04-25T11:19:27.442607+0000","last_clean":"2024-04-25T11:19:27.442607+0000","last_became_active":"2024-04-25T11:19:27.438198+0000","last_became_peered":"2024-04-25T11:19:27.438198+0000","last_unstale":"2024-04-25T11:19:27.442607+0000","last_undegraded":"2024-04-25T11:19:27.442607+0000","last_fullsized":"2024-04-25T11:19:27.442607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.059005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.433601+0000","last_change":"2024-04-25T11:19:27.425325+0000","last_active":"2024-04-25T11:19:27.433601+0000","last_peered":"2024-04-25T11:19:27.433601+0000","last_clean":"2024-04-25T11:19:27.433601+0000","last_became_active":"2024-04-25T11:19:27.423815+0000","last_became_peered":"2024-04-25T11:19:27.423815+0000","last_unstale":"2024-04-25T11:19:27.433601+0000","last_undegraded":"2024-04-25T11:19:27.433601+0000","last_fullsized":"2024-04-25T11:19:27.433601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.963336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818101+0000","last_change":"2024-04-25T11:19:27.447151+0000","last_active":"2024-04-25T11:19:28.818101+0000","last_peered":"2024-04-25T11:19:28.818101+0000","last_clean":"2024-04-25T11:19:28.818101+0000","last_became_active":"2024-04-25T11:19:27.446193+0000","last_became_peered":"2024-04-25T11:19:27.446193+0000","last_unstale":"2024-04-25T11:19:28.818101+0000","last_undegraded":"2024-04-25T11:19:28.818101+0000","last_fullsized":"2024-04-25T11:19:28.818101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:01.196607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424335+0000","last_change":"2024-04-25T11:19:27.440245+0000","last_active":"2024-04-25T11:19:28.424335+0000","last_peered":"2024-04-25T11:19:28.424335+0000","last_clean":"2024-04-25T11:19:28.424335+0000","last_became_active":"2024-04-25T11:19:27.439755+0000","last_became_peered":"2024-04-25T11:19:27.439755+0000","last_unstale":"2024-04-25T11:19:28.424335+0000","last_undegraded":"2024-04-25T11:19:28.424335+0000","last_fullsized":"2024-04-25T11:19:28.424335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433384+0000","last_change":"2024-04-25T11:19:27.433866+0000","last_active":"2024-04-25T11:19:28.433384+0000","last_peered":"2024-04-25T11:19:28.433384+0000","last_clean":"2024-04-25T11:19:28.433384+0000","last_became_active":"2024-04-25T11:19:27.423031+0000","last_became_peered":"2024-04-25T11:19:27.423031+0000","last_unstale":"2024-04-25T11:19:28.433384+0000","last_undegraded":"2024-04-25T11:19:28.433384+0000","last_fullsized":"2024-04-25T11:19:28.433384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.444260+0000","last_change":"2024-04-25T11:19:27.427233+0000","last_active":"2024-04-25T11:19:27.444260+0000","last_peered":"2024-04-25T11:19:27.444260+0000","last_clean":"2024-04-25T11:19:27.444260+0000","last_became_active":"2024-04-25T11:19:27.426438+0000","last_became_peered":"2024-04-25T11:19:27.426438+0000","last_unstale":"2024-04-25T11:19:27.444260+0000","last_undegraded":"2024-04-25T11:19:27.444260+0000","last_fullsized":"2024-04-25T11:19:27.444260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:29.922492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452596+0000","last_change":"2024-04-25T11:19:26.448872+0000","last_active":"2024-04-25T11:19:27.452596+0000","last_peered":"2024-04-25T11:19:27.452596+0000","last_clean":"2024-04-25T11:19:27.452596+0000","last_became_active":"2024-04-25T11:19:26.448492+0000","last_became_peered":"2024-04-25T11:19:26.448492+0000","last_unstale":"2024-04-25T11:19:27.452596+0000","last_undegraded":"2024-04-25T11:19:27.452596+0000","last_fullsized":"2024-04-25T11:19:27.452596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.433577+0000","last_change":"2024-04-25T11:19:27.424636+0000","last_active":"2024-04-25T11:19:27.433577+0000","last_peered":"2024-04-25T11:19:27.433577+0000","last_clean":"2024-04-25T11:19:27.433577+0000","last_became_active":"2024-04-25T11:19:27.422364+0000","last_became_peered":"2024-04-25T11:19:27.422364+0000","last_unstale":"2024-04-25T11:19:27.433577+0000","last_undegraded":"2024-04-25T11:19:27.433577+0000","last_fullsized":"2024-04-25T11:19:27.433577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.750362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432509+0000","last_change":"2024-04-25T11:19:26.446971+0000","last_active":"2024-04-25T11:19:28.432509+0000","last_peered":"2024-04-25T11:19:28.432509+0000","last_clean":"2024-04-25T11:19:28.432509+0000","last_became_active":"2024-04-25T11:19:26.445751+0000","last_became_peered":"2024-04-25T11:19:26.445751+0000","last_unstale":"2024-04-25T11:19:28.432509+0000","last_undegraded":"2024-04-25T11:19:28.432509+0000","last_fullsized":"2024-04-25T11:19:28.432509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.406236+0000","last_change":"2024-04-25T11:19:26.430729+0000","last_active":"2024-04-25T11:19:27.406236+0000","last_peered":"2024-04-25T11:19:27.406236+0000","last_clean":"2024-04-25T11:19:27.406236+0000","last_became_active":"2024-04-25T11:19:26.430240+0000","last_became_peered":"2024-04-25T11:19:26.430240+0000","last_unstale":"2024-04-25T11:19:27.406236+0000","last_undegraded":"2024-04-25T11:19:27.406236+0000","last_fullsized":"2024-04-25T11:19:27.406236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:10.270275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.434260+0000","last_change":"2024-04-25T11:19:27.425985+0000","last_active":"2024-04-25T11:19:27.434260+0000","last_peered":"2024-04-25T11:19:27.434260+0000","last_clean":"2024-04-25T11:19:27.434260+0000","last_became_active":"2024-04-25T11:19:27.424091+0000","last_became_peered":"2024-04-25T11:19:27.424091+0000","last_unstale":"2024-04-25T11:19:27.434260+0000","last_undegraded":"2024-04-25T11:19:27.434260+0000","last_fullsized":"2024-04-25T11:19:27.434260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:30.664977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817641+0000","last_change":"2024-04-25T11:19:26.446399+0000","last_active":"2024-04-25T11:19:28.817641+0000","last_peered":"2024-04-25T11:19:28.817641+0000","last_clean":"2024-04-25T11:19:28.817641+0000","last_became_active":"2024-04-25T11:19:26.445502+0000","last_became_peered":"2024-04-25T11:19:26.445502+0000","last_unstale":"2024-04-25T11:19:28.817641+0000","last_undegraded":"2024-04-25T11:19:28.817641+0000","last_fullsized":"2024-04-25T11:19:28.817641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421333+0000","last_change":"2024-04-25T11:19:27.435289+0000","last_active":"2024-04-25T11:19:28.421333+0000","last_peered":"2024-04-25T11:19:28.421333+0000","last_clean":"2024-04-25T11:19:28.421333+0000","last_became_active":"2024-04-25T11:19:27.434888+0000","last_became_peered":"2024-04-25T11:19:27.434888+0000","last_unstale":"2024-04-25T11:19:28.421333+0000","last_undegraded":"2024-04-25T11:19:28.421333+0000","last_fullsized":"2024-04-25T11:19:28.421333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432966+0000","last_change":"2024-04-25T11:19:26.447470+0000","last_active":"2024-04-25T11:19:28.432966+0000","last_peered":"2024-04-25T11:19:28.432966+0000","last_clean":"2024-04-25T11:19:28.432966+0000","last_became_active":"2024-04-25T11:19:26.445469+0000","last_became_peered":"2024-04-25T11:19:26.445469+0000","last_unstale":"2024-04-25T11:19:28.432966+0000","last_undegraded":"2024-04-25T11:19:28.432966+0000","last_fullsized":"2024-04-25T11:19:28.432966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818172+0000","last_change":"2024-04-25T11:19:27.454986+0000","last_active":"2024-04-25T11:19:28.818172+0000","last_peered":"2024-04-25T11:19:28.818172+0000","last_clean":"2024-04-25T11:19:28.818172+0000","last_became_active":"2024-04-25T11:19:27.448543+0000","last_became_peered":"2024-04-25T11:19:27.448543+0000","last_unstale":"2024-04-25T11:19:28.818172+0000","last_undegraded":"2024-04-25T11:19:28.818172+0000","last_fullsized":"2024-04-25T11:19:28.818172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.329120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818061+0000","last_change":"2024-04-25T11:19:26.446709+0000","last_active":"2024-04-25T11:19:28.818061+0000","last_peered":"2024-04-25T11:19:28.818061+0000","last_clean":"2024-04-25T11:19:28.818061+0000","last_became_active":"2024-04-25T11:19:26.445742+0000","last_became_peered":"2024-04-25T11:19:26.445742+0000","last_unstale":"2024-04-25T11:19:28.818061+0000","last_undegraded":"2024-04-25T11:19:28.818061+0000","last_fullsized":"2024-04-25T11:19:28.818061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.437334+0000","last_change":"2024-04-25T11:19:27.431751+0000","last_active":"2024-04-25T11:19:27.437334+0000","last_peered":"2024-04-25T11:19:27.437334+0000","last_clean":"2024-04-25T11:19:27.437334+0000","last_became_active":"2024-04-25T11:19:27.431063+0000","last_became_peered":"2024-04-25T11:19:27.431063+0000","last_unstale":"2024-04-25T11:19:27.437334+0000","last_undegraded":"2024-04-25T11:19:27.437334+0000","last_fullsized":"2024-04-25T11:19:27.437334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422020+0000","last_change":"2024-04-25T11:19:26.435639+0000","last_active":"2024-04-25T11:19:28.422020+0000","last_peered":"2024-04-25T11:19:28.422020+0000","last_clean":"2024-04-25T11:19:28.422020+0000","last_became_active":"2024-04-25T11:19:26.435051+0000","last_became_peered":"2024-04-25T11:19:26.435051+0000","last_unstale":"2024-04-25T11:19:28.422020+0000","last_undegraded":"2024-04-25T11:19:28.422020+0000","last_fullsized":"2024-04-25T11:19:28.422020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452528+0000","last_change":"2024-04-25T11:19:27.449942+0000","last_active":"2024-04-25T11:19:27.452528+0000","last_peered":"2024-04-25T11:19:27.452528+0000","last_clean":"2024-04-25T11:19:27.452528+0000","last_became_active":"2024-04-25T11:19:27.449419+0000","last_became_peered":"2024-04-25T11:19:27.449419+0000","last_unstale":"2024-04-25T11:19:27.452528+0000","last_undegraded":"2024-04-25T11:19:27.452528+0000","last_fullsized":"2024-04-25T11:19:27.452528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.750362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432820+0000","last_change":"2024-04-25T11:19:26.437548+0000","last_active":"2024-04-25T11:19:28.432820+0000","last_peered":"2024-04-25T11:19:28.432820+0000","last_clean":"2024-04-25T11:19:28.432820+0000","last_became_active":"2024-04-25T11:19:26.436659+0000","last_became_peered":"2024-04-25T11:19:26.436659+0000","last_unstale":"2024-04-25T11:19:28.432820+0000","last_undegraded":"2024-04-25T11:19:28.432820+0000","last_fullsized":"2024-04-25T11:19:28.432820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451464+0000","last_change":"2024-04-25T11:19:27.443841+0000","last_active":"2024-04-25T11:19:27.451464+0000","last_peered":"2024-04-25T11:19:27.451464+0000","last_clean":"2024-04-25T11:19:27.451464+0000","last_became_active":"2024-04-25T11:19:27.443174+0000","last_became_peered":"2024-04-25T11:19:27.443174+0000","last_unstale":"2024-04-25T11:19:27.451464+0000","last_undegraded":"2024-04-25T11:19:27.451464+0000","last_fullsized":"2024-04-25T11:19:27.451464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.443303+0000","last_change":"2024-04-25T11:19:26.444467+0000","last_active":"2024-04-25T11:19:27.443303+0000","last_peered":"2024-04-25T11:19:27.443303+0000","last_clean":"2024-04-25T11:19:27.443303+0000","last_became_active":"2024-04-25T11:19:26.444081+0000","last_became_peered":"2024-04-25T11:19:26.444081+0000","last_unstale":"2024-04-25T11:19:27.443303+0000","last_undegraded":"2024-04-25T11:19:27.443303+0000","last_fullsized":"2024-04-25T11:19:27.443303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:10.270275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.445474+0000","last_change":"2024-04-25T11:19:27.442198+0000","last_active":"2024-04-25T11:19:27.445474+0000","last_peered":"2024-04-25T11:19:27.445474+0000","last_clean":"2024-04-25T11:19:27.445474+0000","last_became_active":"2024-04-25T11:19:27.441771+0000","last_became_peered":"2024-04-25T11:19:27.441771+0000","last_unstale":"2024-04-25T11:19:27.445474+0000","last_undegraded":"2024-04-25T11:19:27.445474+0000","last_fullsized":"2024-04-25T11:19:27.445474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.059005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818060+0000","last_change":"2024-04-25T11:19:26.443383+0000","last_active":"2024-04-25T11:19:28.818060+0000","last_peered":"2024-04-25T11:19:28.818060+0000","last_clean":"2024-04-25T11:19:28.818060+0000","last_became_active":"2024-04-25T11:19:26.442678+0000","last_became_peered":"2024-04-25T11:19:26.442678+0000","last_unstale":"2024-04-25T11:19:28.818060+0000","last_undegraded":"2024-04-25T11:19:28.818060+0000","last_fullsized":"2024-04-25T11:19:28.818060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452806+0000","last_change":"2024-04-25T11:19:27.448069+0000","last_active":"2024-04-25T11:19:27.452806+0000","last_peered":"2024-04-25T11:19:27.452806+0000","last_clean":"2024-04-25T11:19:27.452806+0000","last_became_active":"2024-04-25T11:19:27.447313+0000","last_became_peered":"2024-04-25T11:19:27.447313+0000","last_unstale":"2024-04-25T11:19:27.452806+0000","last_undegraded":"2024-04-25T11:19:27.452806+0000","last_fullsized":"2024-04-25T11:19:27.452806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.942052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432496+0000","last_change":"2024-04-25T11:19:26.449988+0000","last_active":"2024-04-25T11:19:28.432496+0000","last_peered":"2024-04-25T11:19:28.432496+0000","last_clean":"2024-04-25T11:19:28.432496+0000","last_became_active":"2024-04-25T11:19:26.445253+0000","last_became_peered":"2024-04-25T11:19:26.445253+0000","last_unstale":"2024-04-25T11:19:28.432496+0000","last_undegraded":"2024-04-25T11:19:28.432496+0000","last_fullsized":"2024-04-25T11:19:28.432496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432903+0000","last_change":"2024-04-25T11:19:27.423470+0000","last_active":"2024-04-25T11:19:28.432903+0000","last_peered":"2024-04-25T11:19:28.432903+0000","last_clean":"2024-04-25T11:19:28.432903+0000","last_became_active":"2024-04-25T11:19:27.422649+0000","last_became_peered":"2024-04-25T11:19:27.422649+0000","last_unstale":"2024-04-25T11:19:28.432903+0000","last_undegraded":"2024-04-25T11:19:28.432903+0000","last_fullsized":"2024-04-25T11:19:28.432903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.423793+0000","last_change":"2024-04-25T11:19:26.424599+0000","last_active":"2024-04-25T11:19:28.423793+0000","last_peered":"2024-04-25T11:19:28.423793+0000","last_clean":"2024-04-25T11:19:28.423793+0000","last_became_active":"2024-04-25T11:19:26.424110+0000","last_became_peered":"2024-04-25T11:19:26.424110+0000","last_unstale":"2024-04-25T11:19:28.423793+0000","last_undegraded":"2024-04-25T11:19:28.423793+0000","last_fullsized":"2024-04-25T11:19:28.423793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451272+0000","last_change":"2024-04-25T11:19:27.441846+0000","last_active":"2024-04-25T11:19:27.451272+0000","last_peered":"2024-04-25T11:19:27.451272+0000","last_clean":"2024-04-25T11:19:27.451272+0000","last_became_active":"2024-04-25T11:19:27.441349+0000","last_became_peered":"2024-04-25T11:19:27.441349+0000","last_unstale":"2024-04-25T11:19:27.451272+0000","last_undegraded":"2024-04-25T11:19:27.451272+0000","last_fullsized":"2024-04-25T11:19:27.451272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.067341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.443451+0000","last_change":"2024-04-25T11:19:26.429643+0000","last_active":"2024-04-25T11:19:27.443451+0000","last_peered":"2024-04-25T11:19:27.443451+0000","last_clean":"2024-04-25T11:19:27.443451+0000","last_became_active":"2024-04-25T11:19:26.428760+0000","last_became_peered":"2024-04-25T11:19:26.428760+0000","last_unstale":"2024-04-25T11:19:27.443451+0000","last_undegraded":"2024-04-25T11:19:27.443451+0000","last_fullsized":"2024-04-25T11:19:27.443451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:36.804999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452404+0000","last_change":"2024-04-25T11:19:27.445422+0000","last_active":"2024-04-25T11:19:27.452404+0000","last_peered":"2024-04-25T11:19:27.452404+0000","last_clean":"2024-04-25T11:19:27.452404+0000","last_became_active":"2024-04-25T11:19:27.444978+0000","last_became_peered":"2024-04-25T11:19:27.444978+0000","last_unstale":"2024-04-25T11:19:27.452404+0000","last_undegraded":"2024-04-25T11:19:27.452404+0000","last_fullsized":"2024-04-25T11:19:27.452404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424380+0000","last_change":"2024-04-25T11:19:26.415971+0000","last_active":"2024-04-25T11:19:28.424380+0000","last_peered":"2024-04-25T11:19:28.424380+0000","last_clean":"2024-04-25T11:19:28.424380+0000","last_became_active":"2024-04-25T11:19:26.415532+0000","last_became_peered":"2024-04-25T11:19:26.415532+0000","last_unstale":"2024-04-25T11:19:28.424380+0000","last_undegraded":"2024-04-25T11:19:28.424380+0000","last_fullsized":"2024-04-25T11:19:28.424380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452868+0000","last_change":"2024-04-25T11:19:27.445828+0000","last_active":"2024-04-25T11:19:27.452868+0000","last_peered":"2024-04-25T11:19:27.452868+0000","last_clean":"2024-04-25T11:19:27.452868+0000","last_became_active":"2024-04-25T11:19:27.445424+0000","last_became_peered":"2024-04-25T11:19:27.445424+0000","last_unstale":"2024-04-25T11:19:27.452868+0000","last_undegraded":"2024-04-25T11:19:27.452868+0000","last_fullsized":"2024-04-25T11:19:27.452868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422118+0000","last_change":"2024-04-25T11:19:26.442966+0000","last_active":"2024-04-25T11:19:28.422118+0000","last_peered":"2024-04-25T11:19:28.422118+0000","last_clean":"2024-04-25T11:19:28.422118+0000","last_became_active":"2024-04-25T11:19:26.442531+0000","last_became_peered":"2024-04-25T11:19:26.442531+0000","last_unstale":"2024-04-25T11:19:28.422118+0000","last_undegraded":"2024-04-25T11:19:28.422118+0000","last_fullsized":"2024-04-25T11:19:28.422118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818802+0000","last_change":"2024-04-25T11:19:27.454129+0000","last_active":"2024-04-25T11:19:28.818802+0000","last_peered":"2024-04-25T11:19:28.818802+0000","last_clean":"2024-04-25T11:19:28.818802+0000","last_became_active":"2024-04-25T11:19:27.453260+0000","last_became_peered":"2024-04-25T11:19:27.453260+0000","last_unstale":"2024-04-25T11:19:28.818802+0000","last_undegraded":"2024-04-25T11:19:28.818802+0000","last_fullsized":"2024-04-25T11:19:28.818802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.297417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424074+0000","last_change":"2024-04-25T11:19:26.435919+0000","last_active":"2024-04-25T11:19:28.424074+0000","last_peered":"2024-04-25T11:19:28.424074+0000","last_clean":"2024-04-25T11:19:28.424074+0000","last_became_active":"2024-04-25T11:19:26.435172+0000","last_became_peered":"2024-04-25T11:19:26.435172+0000","last_unstale":"2024-04-25T11:19:28.424074+0000","last_undegraded":"2024-04-25T11:19:28.424074+0000","last_fullsized":"2024-04-25T11:19:28.424074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.453714+0000","last_change":"2024-04-25T11:19:27.452401+0000","last_active":"2024-04-25T11:19:27.453714+0000","last_peered":"2024-04-25T11:19:27.453714+0000","last_clean":"2024-04-25T11:19:27.453714+0000","last_became_active":"2024-04-25T11:19:27.452007+0000","last_became_peered":"2024-04-25T11:19:27.452007+0000","last_unstale":"2024-04-25T11:19:27.453714+0000","last_undegraded":"2024-04-25T11:19:27.453714+0000","last_fullsized":"2024-04-25T11:19:27.453714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.398422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":108,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.410291+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:19:27.410291+0000","last_peered":"2024-04-25T11:19:27.410291+0000","last_clean":"2024-04-25T11:19:27.410291+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:19:27.410291+0000","last_undegraded":"2024-04-25T11:19:27.410291+0000","last_fullsized":"2024-04-25T11:19:27.410291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817551+0000","last_change":"2024-04-25T11:19:26.431881+0000","last_active":"2024-04-25T11:19:28.817551+0000","last_peered":"2024-04-25T11:19:28.817551+0000","last_clean":"2024-04-25T11:19:28.817551+0000","last_became_active":"2024-04-25T11:19:26.431224+0000","last_became_peered":"2024-04-25T11:19:26.431224+0000","last_unstale":"2024-04-25T11:19:28.817551+0000","last_undegraded":"2024-04-25T11:19:28.817551+0000","last_fullsized":"2024-04-25T11:19:28.817551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.455043+0000","last_change":"2024-04-25T11:19:27.453756+0000","last_active":"2024-04-25T11:19:27.455043+0000","last_peered":"2024-04-25T11:19:27.455043+0000","last_clean":"2024-04-25T11:19:27.455043+0000","last_became_active":"2024-04-25T11:19:27.453406+0000","last_became_peered":"2024-04-25T11:19:27.453406+0000","last_unstale":"2024-04-25T11:19:27.455043+0000","last_undegraded":"2024-04-25T11:19:27.455043+0000","last_fullsized":"2024-04-25T11:19:27.455043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:41.086001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.424531+0000","last_change":"2024-04-25T11:19:26.439003+0000","last_active":"2024-04-25T11:19:27.424531+0000","last_peered":"2024-04-25T11:19:27.424531+0000","last_clean":"2024-04-25T11:19:27.424531+0000","last_became_active":"2024-04-25T11:19:26.437864+0000","last_became_peered":"2024-04-25T11:19:26.437864+0000","last_unstale":"2024-04-25T11:19:27.424531+0000","last_undegraded":"2024-04-25T11:19:27.424531+0000","last_fullsized":"2024-04-25T11:19:27.424531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:03.260366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.418526+0000","last_change":"2024-04-25T11:19:27.411133+0000","last_active":"2024-04-25T11:19:27.418526+0000","last_peered":"2024-04-25T11:19:27.418526+0000","last_clean":"2024-04-25T11:19:27.418526+0000","last_became_active":"2024-04-25T11:19:27.410234+0000","last_became_peered":"2024-04-25T11:19:27.410234+0000","last_unstale":"2024-04-25T11:19:27.418526+0000","last_undegraded":"2024-04-25T11:19:27.418526+0000","last_fullsized":"2024-04-25T11:19:27.418526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:29.922492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424634+0000","last_change":"2024-04-25T11:19:26.413057+0000","last_active":"2024-04-25T11:19:28.424634+0000","last_peered":"2024-04-25T11:19:28.424634+0000","last_clean":"2024-04-25T11:19:28.424634+0000","last_became_active":"2024-04-25T11:19:26.412586+0000","last_became_peered":"2024-04-25T11:19:26.412586+0000","last_unstale":"2024-04-25T11:19:28.424634+0000","last_undegraded":"2024-04-25T11:19:28.424634+0000","last_fullsized":"2024-04-25T11:19:28.424634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421086+0000","last_change":"2024-04-25T11:19:27.423812+0000","last_active":"2024-04-25T11:19:28.421086+0000","last_peered":"2024-04-25T11:19:28.421086+0000","last_clean":"2024-04-25T11:19:28.421086+0000","last_became_active":"2024-04-25T11:19:27.419444+0000","last_became_peered":"2024-04-25T11:19:27.419444+0000","last_unstale":"2024-04-25T11:19:28.421086+0000","last_undegraded":"2024-04-25T11:19:28.421086+0000","last_fullsized":"2024-04-25T11:19:28.421086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818406+0000","last_change":"2024-04-25T11:19:26.433315+0000","last_active":"2024-04-25T11:19:28.818406+0000","last_peered":"2024-04-25T11:19:28.818406+0000","last_clean":"2024-04-25T11:19:28.818406+0000","last_became_active":"2024-04-25T11:19:26.432639+0000","last_became_peered":"2024-04-25T11:19:26.432639+0000","last_unstale":"2024-04-25T11:19:28.818406+0000","last_undegraded":"2024-04-25T11:19:28.818406+0000","last_fullsized":"2024-04-25T11:19:28.818406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:21.523311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.420161+0000","last_change":"2024-04-25T11:19:27.415292+0000","last_active":"2024-04-25T11:19:27.420161+0000","last_peered":"2024-04-25T11:19:27.420161+0000","last_clean":"2024-04-25T11:19:27.420161+0000","last_became_active":"2024-04-25T11:19:27.414270+0000","last_became_peered":"2024-04-25T11:19:27.414270+0000","last_unstale":"2024-04-25T11:19:27.420161+0000","last_undegraded":"2024-04-25T11:19:27.420161+0000","last_fullsized":"2024-04-25T11:19:27.420161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:11.281250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421617+0000","last_change":"2024-04-25T11:19:26.424289+0000","last_active":"2024-04-25T11:19:28.421617+0000","last_peered":"2024-04-25T11:19:28.421617+0000","last_clean":"2024-04-25T11:19:28.421617+0000","last_became_active":"2024-04-25T11:19:26.423826+0000","last_became_peered":"2024-04-25T11:19:26.423826+0000","last_unstale":"2024-04-25T11:19:28.421617+0000","last_undegraded":"2024-04-25T11:19:28.421617+0000","last_fullsized":"2024-04-25T11:19:28.421617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818628+0000","last_change":"2024-04-25T11:19:27.457615+0000","last_active":"2024-04-25T11:19:28.818628+0000","last_peered":"2024-04-25T11:19:28.818628+0000","last_clean":"2024-04-25T11:19:28.818628+0000","last_became_active":"2024-04-25T11:19:27.447354+0000","last_became_peered":"2024-04-25T11:19:27.447354+0000","last_unstale":"2024-04-25T11:19:28.818628+0000","last_undegraded":"2024-04-25T11:19:28.818628+0000","last_fullsized":"2024-04-25T11:19:28.818628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.518039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.424873+0000","last_change":"2024-04-25T11:19:26.459930+0000","last_active":"2024-04-25T11:19:27.424873+0000","last_peered":"2024-04-25T11:19:27.424873+0000","last_clean":"2024-04-25T11:19:27.424873+0000","last_became_active":"2024-04-25T11:19:26.459292+0000","last_became_peered":"2024-04-25T11:19:26.459292+0000","last_unstale":"2024-04-25T11:19:27.424873+0000","last_undegraded":"2024-04-25T11:19:27.424873+0000","last_fullsized":"2024-04-25T11:19:27.424873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:08.607950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422340+0000","last_change":"2024-04-25T11:19:27.422008+0000","last_active":"2024-04-25T11:19:28.422340+0000","last_peered":"2024-04-25T11:19:28.422340+0000","last_clean":"2024-04-25T11:19:28.422340+0000","last_became_active":"2024-04-25T11:19:27.419929+0000","last_became_peered":"2024-04-25T11:19:27.419929+0000","last_unstale":"2024-04-25T11:19:28.422340+0000","last_undegraded":"2024-04-25T11:19:28.422340+0000","last_fullsized":"2024-04-25T11:19:28.422340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.431242+0000","last_change":"2024-04-25T11:19:26.435682+0000","last_active":"2024-04-25T11:19:28.431242+0000","last_peered":"2024-04-25T11:19:28.431242+0000","last_clean":"2024-04-25T11:19:28.431242+0000","last_became_active":"2024-04-25T11:19:26.435173+0000","last_became_peered":"2024-04-25T11:19:26.435173+0000","last_unstale":"2024-04-25T11:19:28.431242+0000","last_undegraded":"2024-04-25T11:19:28.431242+0000","last_fullsized":"2024-04-25T11:19:28.431242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:49.071318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451207+0000","last_change":"2024-04-25T11:19:27.446933+0000","last_active":"2024-04-25T11:19:27.451207+0000","last_peered":"2024-04-25T11:19:27.451207+0000","last_clean":"2024-04-25T11:19:27.451207+0000","last_became_active":"2024-04-25T11:19:27.446601+0000","last_became_peered":"2024-04-25T11:19:27.446601+0000","last_unstale":"2024-04-25T11:19:27.451207+0000","last_undegraded":"2024-04-25T11:19:27.451207+0000","last_fullsized":"2024-04-25T11:19:27.451207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.618925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433109+0000","last_change":"2024-04-25T11:19:26.451159+0000","last_active":"2024-04-25T11:19:28.433109+0000","last_peered":"2024-04-25T11:19:28.433109+0000","last_clean":"2024-04-25T11:19:28.433109+0000","last_became_active":"2024-04-25T11:19:26.450440+0000","last_became_peered":"2024-04-25T11:19:26.450440+0000","last_unstale":"2024-04-25T11:19:28.433109+0000","last_undegraded":"2024-04-25T11:19:28.433109+0000","last_fullsized":"2024-04-25T11:19:28.433109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.433525+0000","last_change":"2024-04-25T11:19:27.428326+0000","last_active":"2024-04-25T11:19:27.433525+0000","last_peered":"2024-04-25T11:19:27.433525+0000","last_clean":"2024-04-25T11:19:27.433525+0000","last_became_active":"2024-04-25T11:19:27.427906+0000","last_became_peered":"2024-04-25T11:19:27.427906+0000","last_unstale":"2024-04-25T11:19:27.433525+0000","last_undegraded":"2024-04-25T11:19:27.433525+0000","last_fullsized":"2024-04-25T11:19:27.433525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:25.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.405903+0000","last_change":"2024-04-25T11:19:26.447443+0000","last_active":"2024-04-25T11:19:27.405903+0000","last_peered":"2024-04-25T11:19:27.405903+0000","last_clean":"2024-04-25T11:19:27.405903+0000","last_became_active":"2024-04-25T11:19:26.439787+0000","last_became_peered":"2024-04-25T11:19:26.439787+0000","last_unstale":"2024-04-25T11:19:27.405903+0000","last_undegraded":"2024-04-25T11:19:27.405903+0000","last_fullsized":"2024-04-25T11:19:27.405903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:49.071318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.437428+0000","last_change":"2024-04-25T11:19:27.422934+0000","last_active":"2024-04-25T11:19:27.437428+0000","last_peered":"2024-04-25T11:19:27.437428+0000","last_clean":"2024-04-25T11:19:27.437428+0000","last_became_active":"2024-04-25T11:19:27.421387+0000","last_became_peered":"2024-04-25T11:19:27.421387+0000","last_unstale":"2024-04-25T11:19:27.437428+0000","last_undegraded":"2024-04-25T11:19:27.437428+0000","last_fullsized":"2024-04-25T11:19:27.437428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:47.993968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421663+0000","last_change":"2024-04-25T11:19:26.456701+0000","last_active":"2024-04-25T11:19:28.421663+0000","last_peered":"2024-04-25T11:19:28.421663+0000","last_clean":"2024-04-25T11:19:28.421663+0000","last_became_active":"2024-04-25T11:19:26.456252+0000","last_became_peered":"2024-04-25T11:19:26.456252+0000","last_unstale":"2024-04-25T11:19:28.421663+0000","last_undegraded":"2024-04-25T11:19:28.421663+0000","last_fullsized":"2024-04-25T11:19:28.421663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442721+0000","last_change":"2024-04-25T11:19:27.435245+0000","last_active":"2024-04-25T11:19:27.442721+0000","last_peered":"2024-04-25T11:19:27.442721+0000","last_clean":"2024-04-25T11:19:27.442721+0000","last_became_active":"2024-04-25T11:19:27.434609+0000","last_became_peered":"2024-04-25T11:19:27.434609+0000","last_unstale":"2024-04-25T11:19:27.442721+0000","last_undegraded":"2024-04-25T11:19:27.442721+0000","last_fullsized":"2024-04-25T11:19:27.442721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422471+0000","last_change":"2024-04-25T11:19:26.423167+0000","last_active":"2024-04-25T11:19:28.422471+0000","last_peered":"2024-04-25T11:19:28.422471+0000","last_clean":"2024-04-25T11:19:28.422471+0000","last_became_active":"2024-04-25T11:19:26.422397+0000","last_became_peered":"2024-04-25T11:19:26.422397+0000","last_unstale":"2024-04-25T11:19:28.422471+0000","last_undegraded":"2024-04-25T11:19:28.422471+0000","last_fullsized":"2024-04-25T11:19:28.422471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.048030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.432628+0000","last_change":"2024-04-25T11:19:27.426394+0000","last_active":"2024-04-25T11:19:27.432628+0000","last_peered":"2024-04-25T11:19:27.432628+0000","last_clean":"2024-04-25T11:19:27.432628+0000","last_became_active":"2024-04-25T11:19:27.424504+0000","last_became_peered":"2024-04-25T11:19:27.424504+0000","last_unstale":"2024-04-25T11:19:27.432628+0000","last_undegraded":"2024-04-25T11:19:27.432628+0000","last_fullsized":"2024-04-25T11:19:27.432628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.750362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.445666+0000","last_change":"2024-04-25T11:19:26.455784+0000","last_active":"2024-04-25T11:19:27.445666+0000","last_peered":"2024-04-25T11:19:27.445666+0000","last_clean":"2024-04-25T11:19:27.445666+0000","last_became_active":"2024-04-25T11:19:26.455125+0000","last_became_peered":"2024-04-25T11:19:26.455125+0000","last_unstale":"2024-04-25T11:19:27.445666+0000","last_undegraded":"2024-04-25T11:19:27.445666+0000","last_fullsized":"2024-04-25T11:19:27.445666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:40.075026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433197+0000","last_change":"2024-04-25T11:19:27.455131+0000","last_active":"2024-04-25T11:19:28.433197+0000","last_peered":"2024-04-25T11:19:28.433197+0000","last_clean":"2024-04-25T11:19:28.433197+0000","last_became_active":"2024-04-25T11:19:27.454732+0000","last_became_peered":"2024-04-25T11:19:27.454732+0000","last_unstale":"2024-04-25T11:19:28.433197+0000","last_undegraded":"2024-04-25T11:19:28.433197+0000","last_fullsized":"2024-04-25T11:19:28.433197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818302+0000","last_change":"2024-04-25T11:19:27.450900+0000","last_active":"2024-04-25T11:19:28.818302+0000","last_peered":"2024-04-25T11:19:28.818302+0000","last_clean":"2024-04-25T11:19:28.818302+0000","last_became_active":"2024-04-25T11:19:27.447933+0000","last_became_peered":"2024-04-25T11:19:27.447933+0000","last_unstale":"2024-04-25T11:19:28.818302+0000","last_undegraded":"2024-04-25T11:19:28.818302+0000","last_fullsized":"2024-04-25T11:19:28.818302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:03.621864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.461486+0000","last_change":"2024-04-25T11:19:26.456557+0000","last_active":"2024-04-25T11:19:27.461486+0000","last_peered":"2024-04-25T11:19:27.461486+0000","last_clean":"2024-04-25T11:19:27.461486+0000","last_became_active":"2024-04-25T11:19:26.455740+0000","last_became_peered":"2024-04-25T11:19:26.455740+0000","last_unstale":"2024-04-25T11:19:27.461486+0000","last_undegraded":"2024-04-25T11:19:27.461486+0000","last_fullsized":"2024-04-25T11:19:27.461486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:12.867967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818605+0000","last_change":"2024-04-25T11:19:27.440111+0000","last_active":"2024-04-25T11:19:28.818605+0000","last_peered":"2024-04-25T11:19:28.818605+0000","last_clean":"2024-04-25T11:19:28.818605+0000","last_became_active":"2024-04-25T11:19:27.430503+0000","last_became_peered":"2024-04-25T11:19:27.430503+0000","last_unstale":"2024-04-25T11:19:28.818605+0000","last_undegraded":"2024-04-25T11:19:28.818605+0000","last_fullsized":"2024-04-25T11:19:28.818605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:45.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.445898+0000","last_change":"2024-04-25T11:19:26.457866+0000","last_active":"2024-04-25T11:19:27.445898+0000","last_peered":"2024-04-25T11:19:27.445898+0000","last_clean":"2024-04-25T11:19:27.445898+0000","last_became_active":"2024-04-25T11:19:26.457223+0000","last_became_peered":"2024-04-25T11:19:26.457223+0000","last_unstale":"2024-04-25T11:19:27.445898+0000","last_undegraded":"2024-04-25T11:19:27.445898+0000","last_fullsized":"2024-04-25T11:19:27.445898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.048030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451829+0000","last_change":"2024-04-25T11:19:27.448305+0000","last_active":"2024-04-25T11:19:27.451829+0000","last_peered":"2024-04-25T11:19:27.451829+0000","last_clean":"2024-04-25T11:19:27.451829+0000","last_became_active":"2024-04-25T11:19:27.447940+0000","last_became_peered":"2024-04-25T11:19:27.447940+0000","last_unstale":"2024-04-25T11:19:27.451829+0000","last_undegraded":"2024-04-25T11:19:27.451829+0000","last_fullsized":"2024-04-25T11:19:27.451829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:04.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.448599+0000","last_change":"2024-04-25T11:19:26.425844+0000","last_active":"2024-04-25T11:19:27.448599+0000","last_peered":"2024-04-25T11:19:27.448599+0000","last_clean":"2024-04-25T11:19:27.448599+0000","last_became_active":"2024-04-25T11:19:26.425443+0000","last_became_peered":"2024-04-25T11:19:26.425443+0000","last_unstale":"2024-04-25T11:19:27.448599+0000","last_undegraded":"2024-04-25T11:19:27.448599+0000","last_fullsized":"2024-04-25T11:19:27.448599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442448+0000","last_change":"2024-04-25T11:19:27.434918+0000","last_active":"2024-04-25T11:19:27.442448+0000","last_peered":"2024-04-25T11:19:27.442448+0000","last_clean":"2024-04-25T11:19:27.442448+0000","last_became_active":"2024-04-25T11:19:27.434444+0000","last_became_peered":"2024-04-25T11:19:27.434444+0000","last_unstale":"2024-04-25T11:19:27.442448+0000","last_undegraded":"2024-04-25T11:19:27.442448+0000","last_fullsized":"2024-04-25T11:19:27.442448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451869+0000","last_change":"2024-04-25T11:19:26.448774+0000","last_active":"2024-04-25T11:19:27.451869+0000","last_peered":"2024-04-25T11:19:27.451869+0000","last_clean":"2024-04-25T11:19:27.451869+0000","last_became_active":"2024-04-25T11:19:26.448304+0000","last_became_peered":"2024-04-25T11:19:26.448304+0000","last_unstale":"2024-04-25T11:19:27.451869+0000","last_undegraded":"2024-04-25T11:19:27.451869+0000","last_fullsized":"2024-04-25T11:19:27.451869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:19:31.504 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:19:31.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c63134640 1 Processor -- start 2024-04-25T11:19:31.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c63134640 1 -- start start 2024-04-25T11:19:31.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c63134640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c06c5f0 0x7f6c5c160400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c63134640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c5c0b5390 0x7f6c5c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c63134640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c5c0b5ce0 con 0x7f6c5c1639f0 2024-04-25T11:19:31.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c63134640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6c5c0af6a0 con 0x7f6c5c0b5390 2024-04-25T11:19:31.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c63134640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5c0af370 con 0x7f6c5c06c5f0 2024-04-25T11:19:31.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c616d1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6c5c1639f0 0x7f6c5c163e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56966/0 (socket says 172.21.15.106:56966) 2024-04-25T11:19:31.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c616d1640 1 -- 172.21.15.106:0/572006155 learned_addr learned my addr 172.21.15.106:0/572006155 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3117134293 0 0) 0x7f6c4c001120 con 0x7f6c5c1639f0 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5bfff640 1 --2- 172.21.15.106:0/572006155 >> v2:172.21.15.134:3300/0 conn(0x7f6c5c0b5390 0x7f6c5c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c48003530 con 0x7f6c5c1639f0 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c60ed0640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c06c5f0 0x7f6c5c160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2124096527 0 0) 0x7f6c48003530 con 0x7f6c5c1639f0 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c06c5f0 msgr2=0x7f6c5c160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c06c5f0 0x7f6c5c160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 >> v2:172.21.15.134:3300/0 conn(0x7f6c5c0b5390 msgr2=0x7f6c5c0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 --2- 172.21.15.106:0/572006155 >> v2:172.21.15.134:3300/0 conn(0x7f6c5c0b5390 0x7f6c5c0b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c160cc0 con 0x7f6c5c1639f0 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f6c4c001850 con 0x7f6c5c1639f0 2024-04-25T11:19:31.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 >> v1:172.21.15.106:6789/0 conn(0x7f6c5c1639f0 legacy=0x7f6c5c163e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c480039c0 0x7f6c48003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48004810 0x7f6c48004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c06c5f0 0x7f6c48005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c0af6a0 con 0x7f6c480039c0 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c0af370 con 0x7f6c48004810 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.689+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4800b9d0 con 0x7f6c5c06c5f0 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c60ed0640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c480039c0 0x7f6c48003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c5bfff640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48004810 0x7f6c48004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c616d1640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c06c5f0 0x7f6c48005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c60ed0640 1 -- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c06c5f0 msgr2=0x7f6c48005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c60ed0640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c06c5f0 0x7f6c48005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c60ed0640 1 -- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48004810 msgr2=0x7f6c48004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c60ed0640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48004810 0x7f6c48004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c60ed0640 1 -- 172.21.15.106:0/572006155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c48003530 con 0x7f6c480039c0 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c60ed0640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c480039c0 0x7f6c48003e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009f90 tx=0x7f6c5401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658d3c2da0a05e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c540b2070 con 0x7f6c480039c0 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c540a5c20 con 0x7f6c480039c0 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c5b7fe640 1 -- 172.21.15.106:0/572006155 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54009c10 con 0x7f6c480039c0 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c63134640 1 -- 172.21.15.106:0/572006155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c480039c0 msgr2=0x7f6c48003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c63134640 1 --2- 172.21.15.106:0/572006155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c480039c0 0x7f6c48003e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009f90 tx=0x7f6c5401cc30 comp rx=0 tx=0).stop 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c616d1640 1 -- 172.21.15.106:0/572006155 reap_dead start 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c63134640 1 -- 172.21.15.106:0/572006155 shutdown_connections 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c63134640 1 -- 172.21.15.106:0/572006155 >> 172.21.15.106:0/572006155 conn(0x7f6c5c0b78b0 msgr2=0x7f6c5c0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c63134640 1 -- 172.21.15.106:0/572006155 shutdown_connections 2024-04-25T11:19:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.693+0000 7f6c63134640 1 -- 172.21.15.106:0/572006155 wait complete. 2024-04-25T11:19:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c63134640 1 Processor -- start 2024-04-25T11:19:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c63134640 1 -- start start 2024-04-25T11:19:31.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c63134640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c480039c0 0x7f6c5c166d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c63134640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c5c168a00 0x7f6c5c0b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c63134640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c15b900 0x7f6c5c15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c63134640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c16b110 con 0x7f6c5c168a00 2024-04-25T11:19:31.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c63134640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c16af90 con 0x7f6c480039c0 2024-04-25T11:19:31.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c63134640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5c16b290 con 0x7f6c5c15b900 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c5c168a00 0x7f6c5c0b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c5c168a00 0x7f6c5c0b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46096/0 (socket says 172.21.15.106:46096) 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 -- 172.21.15.106:0/1577667284 learned_addr learned my addr 172.21.15.106:0/1577667284 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c616d1640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c15b900 0x7f6c5c15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c60ed0640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c480039c0 0x7f6c5c166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 -- 172.21.15.106:0/1577667284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c15b900 msgr2=0x7f6c5c15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c15b900 0x7f6c5c15bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 -- 172.21.15.106:0/1577667284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c480039c0 msgr2=0x7f6c5c166d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c480039c0 0x7f6c5c166d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 -- 172.21.15.106:0/1577667284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c15c250 con 0x7f6c5c168a00 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c5bfff640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c5c168a00 0x7f6c5c0b2d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6c44011ad0 tx=0x7f6c44011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.697+0000 7f6c597fa640 1 -- 172.21.15.106:0/1577667284 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400fbb0 con 0x7f6c5c168a00 2024-04-25T11:19:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c597fa640 1 -- 172.21.15.106:0/1577667284 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4400fd50 con 0x7f6c5c168a00 2024-04-25T11:19:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c167640 con 0x7f6c5c168a00 2024-04-25T11:19:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c597fa640 1 -- 172.21.15.106:0/1577667284 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44014680 con 0x7f6c5c168a00 2024-04-25T11:19:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c167c00 con 0x7f6c5c168a00 2024-04-25T11:19:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c0744a0 con 0x7f6c5c168a00 2024-04-25T11:19:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c597fa640 1 -- 172.21.15.106:0/1577667284 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6c44012040 con 0x7f6c5c168a00 2024-04-25T11:19:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c597fa640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6c3c06f400 0x7f6c3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c597fa640 1 -- 172.21.15.106:0/1577667284 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6c4408fa00 con 0x7f6c5c168a00 2024-04-25T11:19:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c60ed0640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6c3c06f400 0x7f6c3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.701+0000 7f6c60ed0640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6c3c06f400 0x7f6c3c071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c540af000 tx=0x7f6c540a7040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.705+0000 7f6c597fa640 1 -- 172.21.15.106:0/1577667284 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6c5c15c250 con 0x7f6c5c168a00 2024-04-25T11:19:31.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.869+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c5c0b1090 con 0x7f6c3c06f400 2024-04-25T11:19:31.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.877+0000 7f6c597fa640 1 -- 172.21.15.106:0/1577667284 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251896 (secure 0 0 0) 0x7f6c540fa810 con 0x7f6c3c06f400 2024-04-25T11:19:31.878 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:31.880 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6c3c06f400 msgr2=0x7f6c3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6c3c06f400 0x7f6c3c071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c540af000 tx=0x7f6c540a7040 comp rx=0 tx=0).stop 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c5c168a00 msgr2=0x7f6c5c0b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c5c168a00 0x7f6c5c0b2d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6c44011ad0 tx=0x7f6c44011f90 comp rx=0 tx=0).stop 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 shutdown_connections 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c5c15b900 0x7f6c5c15bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6c3c06f400 0x7f6c3c071910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c5c168a00 0x7f6c5c0b2d20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 --2- 172.21.15.106:0/1577667284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c480039c0 0x7f6c5c166d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 >> 172.21.15.106:0/1577667284 conn(0x7f6c5c0b78b0 msgr2=0x7f6c5c1614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 shutdown_connections 2024-04-25T11:19:31.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:31.881+0000 7f6c63134640 1 -- 172.21.15.106:0/1577667284 wait complete. 2024-04-25T11:19:31.899 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T11:19:30.091646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218968,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755752,"statfs":{"total":773094113280,"available":772869890048,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.705814"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452894+0000","last_change":"2024-04-25T11:19:27.448704+0000","last_active":"2024-04-25T11:19:27.452894+0000","last_peered":"2024-04-25T11:19:27.452894+0000","last_clean":"2024-04-25T11:19:27.452894+0000","last_became_active":"2024-04-25T11:19:27.447782+0000","last_became_peered":"2024-04-25T11:19:27.447782+0000","last_unstale":"2024-04-25T11:19:27.452894+0000","last_undegraded":"2024-04-25T11:19:27.452894+0000","last_fullsized":"2024-04-25T11:19:27.452894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.850268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422162+0000","last_change":"2024-04-25T11:19:27.424499+0000","last_active":"2024-04-25T11:19:28.422162+0000","last_peered":"2024-04-25T11:19:28.422162+0000","last_clean":"2024-04-25T11:19:28.422162+0000","last_became_active":"2024-04-25T11:19:27.424122+0000","last_became_peered":"2024-04-25T11:19:27.424122+0000","last_unstale":"2024-04-25T11:19:28.422162+0000","last_undegraded":"2024-04-25T11:19:28.422162+0000","last_fullsized":"2024-04-25T11:19:28.422162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.429833+0000","last_change":"2024-04-25T11:19:27.422128+0000","last_active":"2024-04-25T11:19:27.429833+0000","last_peered":"2024-04-25T11:19:27.429833+0000","last_clean":"2024-04-25T11:19:27.429833+0000","last_became_active":"2024-04-25T11:19:27.421683+0000","last_became_peered":"2024-04-25T11:19:27.421683+0000","last_unstale":"2024-04-25T11:19:27.429833+0000","last_undegraded":"2024-04-25T11:19:27.429833+0000","last_fullsized":"2024-04-25T11:19:27.429833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:30.664977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818544+0000","last_change":"2024-04-25T11:19:27.439794+0000","last_active":"2024-04-25T11:19:28.818544+0000","last_peered":"2024-04-25T11:19:28.818544+0000","last_clean":"2024-04-25T11:19:28.818544+0000","last_became_active":"2024-04-25T11:19:27.439092+0000","last_became_peered":"2024-04-25T11:19:27.439092+0000","last_unstale":"2024-04-25T11:19:28.818544+0000","last_undegraded":"2024-04-25T11:19:28.818544+0000","last_fullsized":"2024-04-25T11:19:28.818544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:52.757499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452464+0000","last_change":"2024-04-25T11:19:27.447315+0000","last_active":"2024-04-25T11:19:27.452464+0000","last_peered":"2024-04-25T11:19:27.452464+0000","last_clean":"2024-04-25T11:19:27.452464+0000","last_became_active":"2024-04-25T11:19:27.446274+0000","last_became_peered":"2024-04-25T11:19:27.446274+0000","last_unstale":"2024-04-25T11:19:27.452464+0000","last_undegraded":"2024-04-25T11:19:27.452464+0000","last_fullsized":"2024-04-25T11:19:27.452464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817544+0000","last_change":"2024-04-25T11:19:27.446758+0000","last_active":"2024-04-25T11:19:28.817544+0000","last_peered":"2024-04-25T11:19:28.817544+0000","last_clean":"2024-04-25T11:19:28.817544+0000","last_became_active":"2024-04-25T11:19:27.440252+0000","last_became_peered":"2024-04-25T11:19:27.440252+0000","last_unstale":"2024-04-25T11:19:28.817544+0000","last_undegraded":"2024-04-25T11:19:28.817544+0000","last_fullsized":"2024-04-25T11:19:28.817544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:08.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451383+0000","last_change":"2024-04-25T11:19:27.442783+0000","last_active":"2024-04-25T11:19:27.451383+0000","last_peered":"2024-04-25T11:19:27.451383+0000","last_clean":"2024-04-25T11:19:27.451383+0000","last_became_active":"2024-04-25T11:19:27.440913+0000","last_became_peered":"2024-04-25T11:19:27.440913+0000","last_unstale":"2024-04-25T11:19:27.451383+0000","last_undegraded":"2024-04-25T11:19:27.451383+0000","last_fullsized":"2024-04-25T11:19:27.451383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.443767+0000","last_change":"2024-04-25T11:19:27.441574+0000","last_active":"2024-04-25T11:19:27.443767+0000","last_peered":"2024-04-25T11:19:27.443767+0000","last_clean":"2024-04-25T11:19:27.443767+0000","last_became_active":"2024-04-25T11:19:27.441265+0000","last_became_peered":"2024-04-25T11:19:27.441265+0000","last_unstale":"2024-04-25T11:19:27.443767+0000","last_undegraded":"2024-04-25T11:19:27.443767+0000","last_fullsized":"2024-04-25T11:19:27.443767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:04.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.464027+0000","last_change":"2024-04-25T11:19:27.459170+0000","last_active":"2024-04-25T11:19:27.464027+0000","last_peered":"2024-04-25T11:19:27.464027+0000","last_clean":"2024-04-25T11:19:27.464027+0000","last_became_active":"2024-04-25T11:19:27.458755+0000","last_became_peered":"2024-04-25T11:19:27.458755+0000","last_unstale":"2024-04-25T11:19:27.464027+0000","last_undegraded":"2024-04-25T11:19:27.464027+0000","last_fullsized":"2024-04-25T11:19:27.464027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:08.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433030+0000","last_change":"2024-04-25T11:19:27.456078+0000","last_active":"2024-04-25T11:19:28.433030+0000","last_peered":"2024-04-25T11:19:28.433030+0000","last_clean":"2024-04-25T11:19:28.433030+0000","last_became_active":"2024-04-25T11:19:27.455302+0000","last_became_peered":"2024-04-25T11:19:27.455302+0000","last_unstale":"2024-04-25T11:19:28.433030+0000","last_undegraded":"2024-04-25T11:19:28.433030+0000","last_fullsized":"2024-04-25T11:19:28.433030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432803+0000","last_change":"2024-04-25T11:19:27.459132+0000","last_active":"2024-04-25T11:19:28.432803+0000","last_peered":"2024-04-25T11:19:28.432803+0000","last_clean":"2024-04-25T11:19:28.432803+0000","last_became_active":"2024-04-25T11:19:27.458820+0000","last_became_peered":"2024-04-25T11:19:27.458820+0000","last_unstale":"2024-04-25T11:19:28.432803+0000","last_undegraded":"2024-04-25T11:19:28.432803+0000","last_fullsized":"2024-04-25T11:19:28.432803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.067341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442624+0000","last_change":"2024-04-25T11:19:27.439470+0000","last_active":"2024-04-25T11:19:27.442624+0000","last_peered":"2024-04-25T11:19:27.442624+0000","last_clean":"2024-04-25T11:19:27.442624+0000","last_became_active":"2024-04-25T11:19:27.439141+0000","last_became_peered":"2024-04-25T11:19:27.439141+0000","last_unstale":"2024-04-25T11:19:27.442624+0000","last_undegraded":"2024-04-25T11:19:27.442624+0000","last_fullsized":"2024-04-25T11:19:27.442624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:30.664977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.441977+0000","last_change":"2024-04-25T11:19:27.433821+0000","last_active":"2024-04-25T11:19:27.441977+0000","last_peered":"2024-04-25T11:19:27.441977+0000","last_clean":"2024-04-25T11:19:27.441977+0000","last_became_active":"2024-04-25T11:19:27.433427+0000","last_became_peered":"2024-04-25T11:19:27.433427+0000","last_unstale":"2024-04-25T11:19:27.441977+0000","last_undegraded":"2024-04-25T11:19:27.441977+0000","last_fullsized":"2024-04-25T11:19:27.441977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:25.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.434067+0000","last_change":"2024-04-25T11:19:27.428346+0000","last_active":"2024-04-25T11:19:27.434067+0000","last_peered":"2024-04-25T11:19:27.434067+0000","last_clean":"2024-04-25T11:19:27.434067+0000","last_became_active":"2024-04-25T11:19:27.428006+0000","last_became_peered":"2024-04-25T11:19:27.428006+0000","last_unstale":"2024-04-25T11:19:27.434067+0000","last_undegraded":"2024-04-25T11:19:27.434067+0000","last_fullsized":"2024-04-25T11:19:27.434067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421393+0000","last_change":"2024-04-25T11:19:27.428566+0000","last_active":"2024-04-25T11:19:28.421393+0000","last_peered":"2024-04-25T11:19:28.421393+0000","last_clean":"2024-04-25T11:19:28.421393+0000","last_became_active":"2024-04-25T11:19:27.427282+0000","last_became_peered":"2024-04-25T11:19:27.427282+0000","last_unstale":"2024-04-25T11:19:28.421393+0000","last_undegraded":"2024-04-25T11:19:28.421393+0000","last_fullsized":"2024-04-25T11:19:28.421393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817322+0000","last_change":"2024-04-25T11:19:27.448380+0000","last_active":"2024-04-25T11:19:28.817322+0000","last_peered":"2024-04-25T11:19:28.817322+0000","last_clean":"2024-04-25T11:19:28.817322+0000","last_became_active":"2024-04-25T11:19:27.447971+0000","last_became_peered":"2024-04-25T11:19:27.447971+0000","last_unstale":"2024-04-25T11:19:28.817322+0000","last_undegraded":"2024-04-25T11:19:28.817322+0000","last_fullsized":"2024-04-25T11:19:28.817322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.850268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818713+0000","last_change":"2024-04-25T11:19:27.452389+0000","last_active":"2024-04-25T11:19:28.818713+0000","last_peered":"2024-04-25T11:19:28.818713+0000","last_clean":"2024-04-25T11:19:28.818713+0000","last_became_active":"2024-04-25T11:19:27.451910+0000","last_became_peered":"2024-04-25T11:19:27.451910+0000","last_unstale":"2024-04-25T11:19:28.818713+0000","last_undegraded":"2024-04-25T11:19:28.818713+0000","last_fullsized":"2024-04-25T11:19:28.818713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424567+0000","last_change":"2024-04-25T11:19:27.437476+0000","last_active":"2024-04-25T11:19:28.424567+0000","last_peered":"2024-04-25T11:19:28.424567+0000","last_clean":"2024-04-25T11:19:28.424567+0000","last_became_active":"2024-04-25T11:19:27.436712+0000","last_became_peered":"2024-04-25T11:19:27.436712+0000","last_unstale":"2024-04-25T11:19:28.424567+0000","last_undegraded":"2024-04-25T11:19:28.424567+0000","last_fullsized":"2024-04-25T11:19:28.424567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452583+0000","last_change":"2024-04-25T11:19:27.445559+0000","last_active":"2024-04-25T11:19:27.452583+0000","last_peered":"2024-04-25T11:19:27.452583+0000","last_clean":"2024-04-25T11:19:27.452583+0000","last_became_active":"2024-04-25T11:19:27.445058+0000","last_became_peered":"2024-04-25T11:19:27.445058+0000","last_unstale":"2024-04-25T11:19:27.452583+0000","last_undegraded":"2024-04-25T11:19:27.452583+0000","last_fullsized":"2024-04-25T11:19:27.452583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432846+0000","last_change":"2024-04-25T11:19:27.441427+0000","last_active":"2024-04-25T11:19:28.432846+0000","last_peered":"2024-04-25T11:19:28.432846+0000","last_clean":"2024-04-25T11:19:28.432846+0000","last_became_active":"2024-04-25T11:19:27.441087+0000","last_became_peered":"2024-04-25T11:19:27.441087+0000","last_unstale":"2024-04-25T11:19:28.432846+0000","last_undegraded":"2024-04-25T11:19:28.432846+0000","last_fullsized":"2024-04-25T11:19:28.432846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451023+0000","last_change":"2024-04-25T11:19:27.445781+0000","last_active":"2024-04-25T11:19:27.451023+0000","last_peered":"2024-04-25T11:19:27.451023+0000","last_clean":"2024-04-25T11:19:27.451023+0000","last_became_active":"2024-04-25T11:19:27.445240+0000","last_became_peered":"2024-04-25T11:19:27.445240+0000","last_unstale":"2024-04-25T11:19:27.451023+0000","last_undegraded":"2024-04-25T11:19:27.451023+0000","last_fullsized":"2024-04-25T11:19:27.451023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:13.878942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817748+0000","last_change":"2024-04-25T11:19:27.453395+0000","last_active":"2024-04-25T11:19:28.817748+0000","last_peered":"2024-04-25T11:19:28.817748+0000","last_clean":"2024-04-25T11:19:28.817748+0000","last_became_active":"2024-04-25T11:19:27.452942+0000","last_became_peered":"2024-04-25T11:19:27.452942+0000","last_unstale":"2024-04-25T11:19:28.817748+0000","last_undegraded":"2024-04-25T11:19:28.817748+0000","last_fullsized":"2024-04-25T11:19:28.817748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:39.163529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.434048+0000","last_change":"2024-04-25T11:19:27.427559+0000","last_active":"2024-04-25T11:19:27.434048+0000","last_peered":"2024-04-25T11:19:27.434048+0000","last_clean":"2024-04-25T11:19:27.434048+0000","last_became_active":"2024-04-25T11:19:27.421099+0000","last_became_peered":"2024-04-25T11:19:27.421099+0000","last_unstale":"2024-04-25T11:19:27.434048+0000","last_undegraded":"2024-04-25T11:19:27.434048+0000","last_fullsized":"2024-04-25T11:19:27.434048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.963336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817459+0000","last_change":"2024-04-25T11:19:27.440162+0000","last_active":"2024-04-25T11:19:28.817459+0000","last_peered":"2024-04-25T11:19:28.817459+0000","last_clean":"2024-04-25T11:19:28.817459+0000","last_became_active":"2024-04-25T11:19:27.439384+0000","last_became_peered":"2024-04-25T11:19:27.439384+0000","last_unstale":"2024-04-25T11:19:28.817459+0000","last_undegraded":"2024-04-25T11:19:28.817459+0000","last_fullsized":"2024-04-25T11:19:28.817459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.398422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442680+0000","last_change":"2024-04-25T11:19:27.438911+0000","last_active":"2024-04-25T11:19:27.442680+0000","last_peered":"2024-04-25T11:19:27.442680+0000","last_clean":"2024-04-25T11:19:27.442680+0000","last_became_active":"2024-04-25T11:19:27.437950+0000","last_became_peered":"2024-04-25T11:19:27.437950+0000","last_unstale":"2024-04-25T11:19:27.442680+0000","last_undegraded":"2024-04-25T11:19:27.442680+0000","last_fullsized":"2024-04-25T11:19:27.442680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:13.878942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433178+0000","last_change":"2024-04-25T11:19:27.447613+0000","last_active":"2024-04-25T11:19:28.433178+0000","last_peered":"2024-04-25T11:19:28.433178+0000","last_clean":"2024-04-25T11:19:28.433178+0000","last_became_active":"2024-04-25T11:19:27.447314+0000","last_became_peered":"2024-04-25T11:19:27.447314+0000","last_unstale":"2024-04-25T11:19:28.433178+0000","last_undegraded":"2024-04-25T11:19:28.433178+0000","last_fullsized":"2024-04-25T11:19:28.433178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421840+0000","last_change":"2024-04-25T11:19:27.436767+0000","last_active":"2024-04-25T11:19:28.421840+0000","last_peered":"2024-04-25T11:19:28.421840+0000","last_clean":"2024-04-25T11:19:28.421840+0000","last_became_active":"2024-04-25T11:19:27.436392+0000","last_became_peered":"2024-04-25T11:19:27.436392+0000","last_unstale":"2024-04-25T11:19:28.421840+0000","last_undegraded":"2024-04-25T11:19:28.421840+0000","last_fullsized":"2024-04-25T11:19:28.421840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442607+0000","last_change":"2024-04-25T11:19:27.438536+0000","last_active":"2024-04-25T11:19:27.442607+0000","last_peered":"2024-04-25T11:19:27.442607+0000","last_clean":"2024-04-25T11:19:27.442607+0000","last_became_active":"2024-04-25T11:19:27.438198+0000","last_became_peered":"2024-04-25T11:19:27.438198+0000","last_unstale":"2024-04-25T11:19:27.442607+0000","last_undegraded":"2024-04-25T11:19:27.442607+0000","last_fullsized":"2024-04-25T11:19:27.442607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.059005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.433601+0000","last_change":"2024-04-25T11:19:27.425325+0000","last_active":"2024-04-25T11:19:27.433601+0000","last_peered":"2024-04-25T11:19:27.433601+0000","last_clean":"2024-04-25T11:19:27.433601+0000","last_became_active":"2024-04-25T11:19:27.423815+0000","last_became_peered":"2024-04-25T11:19:27.423815+0000","last_unstale":"2024-04-25T11:19:27.433601+0000","last_undegraded":"2024-04-25T11:19:27.433601+0000","last_fullsized":"2024-04-25T11:19:27.433601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.963336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818101+0000","last_change":"2024-04-25T11:19:27.447151+0000","last_active":"2024-04-25T11:19:28.818101+0000","last_peered":"2024-04-25T11:19:28.818101+0000","last_clean":"2024-04-25T11:19:28.818101+0000","last_became_active":"2024-04-25T11:19:27.446193+0000","last_became_peered":"2024-04-25T11:19:27.446193+0000","last_unstale":"2024-04-25T11:19:28.818101+0000","last_undegraded":"2024-04-25T11:19:28.818101+0000","last_fullsized":"2024-04-25T11:19:28.818101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:01.196607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424335+0000","last_change":"2024-04-25T11:19:27.440245+0000","last_active":"2024-04-25T11:19:28.424335+0000","last_peered":"2024-04-25T11:19:28.424335+0000","last_clean":"2024-04-25T11:19:28.424335+0000","last_became_active":"2024-04-25T11:19:27.439755+0000","last_became_peered":"2024-04-25T11:19:27.439755+0000","last_unstale":"2024-04-25T11:19:28.424335+0000","last_undegraded":"2024-04-25T11:19:28.424335+0000","last_fullsized":"2024-04-25T11:19:28.424335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433384+0000","last_change":"2024-04-25T11:19:27.433866+0000","last_active":"2024-04-25T11:19:28.433384+0000","last_peered":"2024-04-25T11:19:28.433384+0000","last_clean":"2024-04-25T11:19:28.433384+0000","last_became_active":"2024-04-25T11:19:27.423031+0000","last_became_peered":"2024-04-25T11:19:27.423031+0000","last_unstale":"2024-04-25T11:19:28.433384+0000","last_undegraded":"2024-04-25T11:19:28.433384+0000","last_fullsized":"2024-04-25T11:19:28.433384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.444260+0000","last_change":"2024-04-25T11:19:27.427233+0000","last_active":"2024-04-25T11:19:27.444260+0000","last_peered":"2024-04-25T11:19:27.444260+0000","last_clean":"2024-04-25T11:19:27.444260+0000","last_became_active":"2024-04-25T11:19:27.426438+0000","last_became_peered":"2024-04-25T11:19:27.426438+0000","last_unstale":"2024-04-25T11:19:27.444260+0000","last_undegraded":"2024-04-25T11:19:27.444260+0000","last_fullsized":"2024-04-25T11:19:27.444260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:29.922492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452596+0000","last_change":"2024-04-25T11:19:26.448872+0000","last_active":"2024-04-25T11:19:27.452596+0000","last_peered":"2024-04-25T11:19:27.452596+0000","last_clean":"2024-04-25T11:19:27.452596+0000","last_became_active":"2024-04-25T11:19:26.448492+0000","last_became_peered":"2024-04-25T11:19:26.448492+0000","last_unstale":"2024-04-25T11:19:27.452596+0000","last_undegraded":"2024-04-25T11:19:27.452596+0000","last_fullsized":"2024-04-25T11:19:27.452596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.433577+0000","last_change":"2024-04-25T11:19:27.424636+0000","last_active":"2024-04-25T11:19:27.433577+0000","last_peered":"2024-04-25T11:19:27.433577+0000","last_clean":"2024-04-25T11:19:27.433577+0000","last_became_active":"2024-04-25T11:19:27.422364+0000","last_became_peered":"2024-04-25T11:19:27.422364+0000","last_unstale":"2024-04-25T11:19:27.433577+0000","last_undegraded":"2024-04-25T11:19:27.433577+0000","last_fullsized":"2024-04-25T11:19:27.433577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.750362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432509+0000","last_change":"2024-04-25T11:19:26.446971+0000","last_active":"2024-04-25T11:19:28.432509+0000","last_peered":"2024-04-25T11:19:28.432509+0000","last_clean":"2024-04-25T11:19:28.432509+0000","last_became_active":"2024-04-25T11:19:26.445751+0000","last_became_peered":"2024-04-25T11:19:26.445751+0000","last_unstale":"2024-04-25T11:19:28.432509+0000","last_undegraded":"2024-04-25T11:19:28.432509+0000","last_fullsized":"2024-04-25T11:19:28.432509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.406236+0000","last_change":"2024-04-25T11:19:26.430729+0000","last_active":"2024-04-25T11:19:27.406236+0000","last_peered":"2024-04-25T11:19:27.406236+0000","last_clean":"2024-04-25T11:19:27.406236+0000","last_became_active":"2024-04-25T11:19:26.430240+0000","last_became_peered":"2024-04-25T11:19:26.430240+0000","last_unstale":"2024-04-25T11:19:27.406236+0000","last_undegraded":"2024-04-25T11:19:27.406236+0000","last_fullsized":"2024-04-25T11:19:27.406236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:10.270275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.434260+0000","last_change":"2024-04-25T11:19:27.425985+0000","last_active":"2024-04-25T11:19:27.434260+0000","last_peered":"2024-04-25T11:19:27.434260+0000","last_clean":"2024-04-25T11:19:27.434260+0000","last_became_active":"2024-04-25T11:19:27.424091+0000","last_became_peered":"2024-04-25T11:19:27.424091+0000","last_unstale":"2024-04-25T11:19:27.434260+0000","last_undegraded":"2024-04-25T11:19:27.434260+0000","last_fullsized":"2024-04-25T11:19:27.434260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:30.664977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817641+0000","last_change":"2024-04-25T11:19:26.446399+0000","last_active":"2024-04-25T11:19:28.817641+0000","last_peered":"2024-04-25T11:19:28.817641+0000","last_clean":"2024-04-25T11:19:28.817641+0000","last_became_active":"2024-04-25T11:19:26.445502+0000","last_became_peered":"2024-04-25T11:19:26.445502+0000","last_unstale":"2024-04-25T11:19:28.817641+0000","last_undegraded":"2024-04-25T11:19:28.817641+0000","last_fullsized":"2024-04-25T11:19:28.817641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421333+0000","last_change":"2024-04-25T11:19:27.435289+0000","last_active":"2024-04-25T11:19:28.421333+0000","last_peered":"2024-04-25T11:19:28.421333+0000","last_clean":"2024-04-25T11:19:28.421333+0000","last_became_active":"2024-04-25T11:19:27.434888+0000","last_became_peered":"2024-04-25T11:19:27.434888+0000","last_unstale":"2024-04-25T11:19:28.421333+0000","last_undegraded":"2024-04-25T11:19:28.421333+0000","last_fullsized":"2024-04-25T11:19:28.421333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432966+0000","last_change":"2024-04-25T11:19:26.447470+0000","last_active":"2024-04-25T11:19:28.432966+0000","last_peered":"2024-04-25T11:19:28.432966+0000","last_clean":"2024-04-25T11:19:28.432966+0000","last_became_active":"2024-04-25T11:19:26.445469+0000","last_became_peered":"2024-04-25T11:19:26.445469+0000","last_unstale":"2024-04-25T11:19:28.432966+0000","last_undegraded":"2024-04-25T11:19:28.432966+0000","last_fullsized":"2024-04-25T11:19:28.432966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818172+0000","last_change":"2024-04-25T11:19:27.454986+0000","last_active":"2024-04-25T11:19:28.818172+0000","last_peered":"2024-04-25T11:19:28.818172+0000","last_clean":"2024-04-25T11:19:28.818172+0000","last_became_active":"2024-04-25T11:19:27.448543+0000","last_became_peered":"2024-04-25T11:19:27.448543+0000","last_unstale":"2024-04-25T11:19:28.818172+0000","last_undegraded":"2024-04-25T11:19:28.818172+0000","last_fullsized":"2024-04-25T11:19:28.818172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.329120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818061+0000","last_change":"2024-04-25T11:19:26.446709+0000","last_active":"2024-04-25T11:19:28.818061+0000","last_peered":"2024-04-25T11:19:28.818061+0000","last_clean":"2024-04-25T11:19:28.818061+0000","last_became_active":"2024-04-25T11:19:26.445742+0000","last_became_peered":"2024-04-25T11:19:26.445742+0000","last_unstale":"2024-04-25T11:19:28.818061+0000","last_undegraded":"2024-04-25T11:19:28.818061+0000","last_fullsized":"2024-04-25T11:19:28.818061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.437334+0000","last_change":"2024-04-25T11:19:27.431751+0000","last_active":"2024-04-25T11:19:27.437334+0000","last_peered":"2024-04-25T11:19:27.437334+0000","last_clean":"2024-04-25T11:19:27.437334+0000","last_became_active":"2024-04-25T11:19:27.431063+0000","last_became_peered":"2024-04-25T11:19:27.431063+0000","last_unstale":"2024-04-25T11:19:27.437334+0000","last_undegraded":"2024-04-25T11:19:27.437334+0000","last_fullsized":"2024-04-25T11:19:27.437334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422020+0000","last_change":"2024-04-25T11:19:26.435639+0000","last_active":"2024-04-25T11:19:28.422020+0000","last_peered":"2024-04-25T11:19:28.422020+0000","last_clean":"2024-04-25T11:19:28.422020+0000","last_became_active":"2024-04-25T11:19:26.435051+0000","last_became_peered":"2024-04-25T11:19:26.435051+0000","last_unstale":"2024-04-25T11:19:28.422020+0000","last_undegraded":"2024-04-25T11:19:28.422020+0000","last_fullsized":"2024-04-25T11:19:28.422020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452528+0000","last_change":"2024-04-25T11:19:27.449942+0000","last_active":"2024-04-25T11:19:27.452528+0000","last_peered":"2024-04-25T11:19:27.452528+0000","last_clean":"2024-04-25T11:19:27.452528+0000","last_became_active":"2024-04-25T11:19:27.449419+0000","last_became_peered":"2024-04-25T11:19:27.449419+0000","last_unstale":"2024-04-25T11:19:27.452528+0000","last_undegraded":"2024-04-25T11:19:27.452528+0000","last_fullsized":"2024-04-25T11:19:27.452528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.750362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432820+0000","last_change":"2024-04-25T11:19:26.437548+0000","last_active":"2024-04-25T11:19:28.432820+0000","last_peered":"2024-04-25T11:19:28.432820+0000","last_clean":"2024-04-25T11:19:28.432820+0000","last_became_active":"2024-04-25T11:19:26.436659+0000","last_became_peered":"2024-04-25T11:19:26.436659+0000","last_unstale":"2024-04-25T11:19:28.432820+0000","last_undegraded":"2024-04-25T11:19:28.432820+0000","last_fullsized":"2024-04-25T11:19:28.432820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451464+0000","last_change":"2024-04-25T11:19:27.443841+0000","last_active":"2024-04-25T11:19:27.451464+0000","last_peered":"2024-04-25T11:19:27.451464+0000","last_clean":"2024-04-25T11:19:27.451464+0000","last_became_active":"2024-04-25T11:19:27.443174+0000","last_became_peered":"2024-04-25T11:19:27.443174+0000","last_unstale":"2024-04-25T11:19:27.451464+0000","last_undegraded":"2024-04-25T11:19:27.451464+0000","last_fullsized":"2024-04-25T11:19:27.451464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.443303+0000","last_change":"2024-04-25T11:19:26.444467+0000","last_active":"2024-04-25T11:19:27.443303+0000","last_peered":"2024-04-25T11:19:27.443303+0000","last_clean":"2024-04-25T11:19:27.443303+0000","last_became_active":"2024-04-25T11:19:26.444081+0000","last_became_peered":"2024-04-25T11:19:26.444081+0000","last_unstale":"2024-04-25T11:19:27.443303+0000","last_undegraded":"2024-04-25T11:19:27.443303+0000","last_fullsized":"2024-04-25T11:19:27.443303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:10.270275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.445474+0000","last_change":"2024-04-25T11:19:27.442198+0000","last_active":"2024-04-25T11:19:27.445474+0000","last_peered":"2024-04-25T11:19:27.445474+0000","last_clean":"2024-04-25T11:19:27.445474+0000","last_became_active":"2024-04-25T11:19:27.441771+0000","last_became_peered":"2024-04-25T11:19:27.441771+0000","last_unstale":"2024-04-25T11:19:27.445474+0000","last_undegraded":"2024-04-25T11:19:27.445474+0000","last_fullsized":"2024-04-25T11:19:27.445474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.059005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818060+0000","last_change":"2024-04-25T11:19:26.443383+0000","last_active":"2024-04-25T11:19:28.818060+0000","last_peered":"2024-04-25T11:19:28.818060+0000","last_clean":"2024-04-25T11:19:28.818060+0000","last_became_active":"2024-04-25T11:19:26.442678+0000","last_became_peered":"2024-04-25T11:19:26.442678+0000","last_unstale":"2024-04-25T11:19:28.818060+0000","last_undegraded":"2024-04-25T11:19:28.818060+0000","last_fullsized":"2024-04-25T11:19:28.818060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452806+0000","last_change":"2024-04-25T11:19:27.448069+0000","last_active":"2024-04-25T11:19:27.452806+0000","last_peered":"2024-04-25T11:19:27.452806+0000","last_clean":"2024-04-25T11:19:27.452806+0000","last_became_active":"2024-04-25T11:19:27.447313+0000","last_became_peered":"2024-04-25T11:19:27.447313+0000","last_unstale":"2024-04-25T11:19:27.452806+0000","last_undegraded":"2024-04-25T11:19:27.452806+0000","last_fullsized":"2024-04-25T11:19:27.452806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.942052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432496+0000","last_change":"2024-04-25T11:19:26.449988+0000","last_active":"2024-04-25T11:19:28.432496+0000","last_peered":"2024-04-25T11:19:28.432496+0000","last_clean":"2024-04-25T11:19:28.432496+0000","last_became_active":"2024-04-25T11:19:26.445253+0000","last_became_peered":"2024-04-25T11:19:26.445253+0000","last_unstale":"2024-04-25T11:19:28.432496+0000","last_undegraded":"2024-04-25T11:19:28.432496+0000","last_fullsized":"2024-04-25T11:19:28.432496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432903+0000","last_change":"2024-04-25T11:19:27.423470+0000","last_active":"2024-04-25T11:19:28.432903+0000","last_peered":"2024-04-25T11:19:28.432903+0000","last_clean":"2024-04-25T11:19:28.432903+0000","last_became_active":"2024-04-25T11:19:27.422649+0000","last_became_peered":"2024-04-25T11:19:27.422649+0000","last_unstale":"2024-04-25T11:19:28.432903+0000","last_undegraded":"2024-04-25T11:19:28.432903+0000","last_fullsized":"2024-04-25T11:19:28.432903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.423793+0000","last_change":"2024-04-25T11:19:26.424599+0000","last_active":"2024-04-25T11:19:28.423793+0000","last_peered":"2024-04-25T11:19:28.423793+0000","last_clean":"2024-04-25T11:19:28.423793+0000","last_became_active":"2024-04-25T11:19:26.424110+0000","last_became_peered":"2024-04-25T11:19:26.424110+0000","last_unstale":"2024-04-25T11:19:28.423793+0000","last_undegraded":"2024-04-25T11:19:28.423793+0000","last_fullsized":"2024-04-25T11:19:28.423793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451272+0000","last_change":"2024-04-25T11:19:27.441846+0000","last_active":"2024-04-25T11:19:27.451272+0000","last_peered":"2024-04-25T11:19:27.451272+0000","last_clean":"2024-04-25T11:19:27.451272+0000","last_became_active":"2024-04-25T11:19:27.441349+0000","last_became_peered":"2024-04-25T11:19:27.441349+0000","last_unstale":"2024-04-25T11:19:27.451272+0000","last_undegraded":"2024-04-25T11:19:27.451272+0000","last_fullsized":"2024-04-25T11:19:27.451272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.067341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.443451+0000","last_change":"2024-04-25T11:19:26.429643+0000","last_active":"2024-04-25T11:19:27.443451+0000","last_peered":"2024-04-25T11:19:27.443451+0000","last_clean":"2024-04-25T11:19:27.443451+0000","last_became_active":"2024-04-25T11:19:26.428760+0000","last_became_peered":"2024-04-25T11:19:26.428760+0000","last_unstale":"2024-04-25T11:19:27.443451+0000","last_undegraded":"2024-04-25T11:19:27.443451+0000","last_fullsized":"2024-04-25T11:19:27.443451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:36.804999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452404+0000","last_change":"2024-04-25T11:19:27.445422+0000","last_active":"2024-04-25T11:19:27.452404+0000","last_peered":"2024-04-25T11:19:27.452404+0000","last_clean":"2024-04-25T11:19:27.452404+0000","last_became_active":"2024-04-25T11:19:27.444978+0000","last_became_peered":"2024-04-25T11:19:27.444978+0000","last_unstale":"2024-04-25T11:19:27.452404+0000","last_undegraded":"2024-04-25T11:19:27.452404+0000","last_fullsized":"2024-04-25T11:19:27.452404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424380+0000","last_change":"2024-04-25T11:19:26.415971+0000","last_active":"2024-04-25T11:19:28.424380+0000","last_peered":"2024-04-25T11:19:28.424380+0000","last_clean":"2024-04-25T11:19:28.424380+0000","last_became_active":"2024-04-25T11:19:26.415532+0000","last_became_peered":"2024-04-25T11:19:26.415532+0000","last_unstale":"2024-04-25T11:19:28.424380+0000","last_undegraded":"2024-04-25T11:19:28.424380+0000","last_fullsized":"2024-04-25T11:19:28.424380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.452868+0000","last_change":"2024-04-25T11:19:27.445828+0000","last_active":"2024-04-25T11:19:27.452868+0000","last_peered":"2024-04-25T11:19:27.452868+0000","last_clean":"2024-04-25T11:19:27.452868+0000","last_became_active":"2024-04-25T11:19:27.445424+0000","last_became_peered":"2024-04-25T11:19:27.445424+0000","last_unstale":"2024-04-25T11:19:27.452868+0000","last_undegraded":"2024-04-25T11:19:27.452868+0000","last_fullsized":"2024-04-25T11:19:27.452868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422118+0000","last_change":"2024-04-25T11:19:26.442966+0000","last_active":"2024-04-25T11:19:28.422118+0000","last_peered":"2024-04-25T11:19:28.422118+0000","last_clean":"2024-04-25T11:19:28.422118+0000","last_became_active":"2024-04-25T11:19:26.442531+0000","last_became_peered":"2024-04-25T11:19:26.442531+0000","last_unstale":"2024-04-25T11:19:28.422118+0000","last_undegraded":"2024-04-25T11:19:28.422118+0000","last_fullsized":"2024-04-25T11:19:28.422118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818802+0000","last_change":"2024-04-25T11:19:27.454129+0000","last_active":"2024-04-25T11:19:28.818802+0000","last_peered":"2024-04-25T11:19:28.818802+0000","last_clean":"2024-04-25T11:19:28.818802+0000","last_became_active":"2024-04-25T11:19:27.453260+0000","last_became_peered":"2024-04-25T11:19:27.453260+0000","last_unstale":"2024-04-25T11:19:28.818802+0000","last_undegraded":"2024-04-25T11:19:28.818802+0000","last_fullsized":"2024-04-25T11:19:28.818802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.297417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424074+0000","last_change":"2024-04-25T11:19:26.435919+0000","last_active":"2024-04-25T11:19:28.424074+0000","last_peered":"2024-04-25T11:19:28.424074+0000","last_clean":"2024-04-25T11:19:28.424074+0000","last_became_active":"2024-04-25T11:19:26.435172+0000","last_became_peered":"2024-04-25T11:19:26.435172+0000","last_unstale":"2024-04-25T11:19:28.424074+0000","last_undegraded":"2024-04-25T11:19:28.424074+0000","last_fullsized":"2024-04-25T11:19:28.424074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.453714+0000","last_change":"2024-04-25T11:19:27.452401+0000","last_active":"2024-04-25T11:19:27.453714+0000","last_peered":"2024-04-25T11:19:27.453714+0000","last_clean":"2024-04-25T11:19:27.453714+0000","last_became_active":"2024-04-25T11:19:27.452007+0000","last_became_peered":"2024-04-25T11:19:27.452007+0000","last_unstale":"2024-04-25T11:19:27.453714+0000","last_undegraded":"2024-04-25T11:19:27.453714+0000","last_fullsized":"2024-04-25T11:19:27.453714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.398422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":108,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.410291+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:19:27.410291+0000","last_peered":"2024-04-25T11:19:27.410291+0000","last_clean":"2024-04-25T11:19:27.410291+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:19:27.410291+0000","last_undegraded":"2024-04-25T11:19:27.410291+0000","last_fullsized":"2024-04-25T11:19:27.410291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817551+0000","last_change":"2024-04-25T11:19:26.431881+0000","last_active":"2024-04-25T11:19:28.817551+0000","last_peered":"2024-04-25T11:19:28.817551+0000","last_clean":"2024-04-25T11:19:28.817551+0000","last_became_active":"2024-04-25T11:19:26.431224+0000","last_became_peered":"2024-04-25T11:19:26.431224+0000","last_unstale":"2024-04-25T11:19:28.817551+0000","last_undegraded":"2024-04-25T11:19:28.817551+0000","last_fullsized":"2024-04-25T11:19:28.817551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.455043+0000","last_change":"2024-04-25T11:19:27.453756+0000","last_active":"2024-04-25T11:19:27.455043+0000","last_peered":"2024-04-25T11:19:27.455043+0000","last_clean":"2024-04-25T11:19:27.455043+0000","last_became_active":"2024-04-25T11:19:27.453406+0000","last_became_peered":"2024-04-25T11:19:27.453406+0000","last_unstale":"2024-04-25T11:19:27.455043+0000","last_undegraded":"2024-04-25T11:19:27.455043+0000","last_fullsized":"2024-04-25T11:19:27.455043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:41.086001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.424531+0000","last_change":"2024-04-25T11:19:26.439003+0000","last_active":"2024-04-25T11:19:27.424531+0000","last_peered":"2024-04-25T11:19:27.424531+0000","last_clean":"2024-04-25T11:19:27.424531+0000","last_became_active":"2024-04-25T11:19:26.437864+0000","last_became_peered":"2024-04-25T11:19:26.437864+0000","last_unstale":"2024-04-25T11:19:27.424531+0000","last_undegraded":"2024-04-25T11:19:27.424531+0000","last_fullsized":"2024-04-25T11:19:27.424531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:03.260366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.418526+0000","last_change":"2024-04-25T11:19:27.411133+0000","last_active":"2024-04-25T11:19:27.418526+0000","last_peered":"2024-04-25T11:19:27.418526+0000","last_clean":"2024-04-25T11:19:27.418526+0000","last_became_active":"2024-04-25T11:19:27.410234+0000","last_became_peered":"2024-04-25T11:19:27.410234+0000","last_unstale":"2024-04-25T11:19:27.418526+0000","last_undegraded":"2024-04-25T11:19:27.418526+0000","last_fullsized":"2024-04-25T11:19:27.418526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:29.922492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424634+0000","last_change":"2024-04-25T11:19:26.413057+0000","last_active":"2024-04-25T11:19:28.424634+0000","last_peered":"2024-04-25T11:19:28.424634+0000","last_clean":"2024-04-25T11:19:28.424634+0000","last_became_active":"2024-04-25T11:19:26.412586+0000","last_became_peered":"2024-04-25T11:19:26.412586+0000","last_unstale":"2024-04-25T11:19:28.424634+0000","last_undegraded":"2024-04-25T11:19:28.424634+0000","last_fullsized":"2024-04-25T11:19:28.424634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421086+0000","last_change":"2024-04-25T11:19:27.423812+0000","last_active":"2024-04-25T11:19:28.421086+0000","last_peered":"2024-04-25T11:19:28.421086+0000","last_clean":"2024-04-25T11:19:28.421086+0000","last_became_active":"2024-04-25T11:19:27.419444+0000","last_became_peered":"2024-04-25T11:19:27.419444+0000","last_unstale":"2024-04-25T11:19:28.421086+0000","last_undegraded":"2024-04-25T11:19:28.421086+0000","last_fullsized":"2024-04-25T11:19:28.421086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818406+0000","last_change":"2024-04-25T11:19:26.433315+0000","last_active":"2024-04-25T11:19:28.818406+0000","last_peered":"2024-04-25T11:19:28.818406+0000","last_clean":"2024-04-25T11:19:28.818406+0000","last_became_active":"2024-04-25T11:19:26.432639+0000","last_became_peered":"2024-04-25T11:19:26.432639+0000","last_unstale":"2024-04-25T11:19:28.818406+0000","last_undegraded":"2024-04-25T11:19:28.818406+0000","last_fullsized":"2024-04-25T11:19:28.818406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:21.523311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.420161+0000","last_change":"2024-04-25T11:19:27.415292+0000","last_active":"2024-04-25T11:19:27.420161+0000","last_peered":"2024-04-25T11:19:27.420161+0000","last_clean":"2024-04-25T11:19:27.420161+0000","last_became_active":"2024-04-25T11:19:27.414270+0000","last_became_peered":"2024-04-25T11:19:27.414270+0000","last_unstale":"2024-04-25T11:19:27.420161+0000","last_undegraded":"2024-04-25T11:19:27.420161+0000","last_fullsized":"2024-04-25T11:19:27.420161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:11.281250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421617+0000","last_change":"2024-04-25T11:19:26.424289+0000","last_active":"2024-04-25T11:19:28.421617+0000","last_peered":"2024-04-25T11:19:28.421617+0000","last_clean":"2024-04-25T11:19:28.421617+0000","last_became_active":"2024-04-25T11:19:26.423826+0000","last_became_peered":"2024-04-25T11:19:26.423826+0000","last_unstale":"2024-04-25T11:19:28.421617+0000","last_undegraded":"2024-04-25T11:19:28.421617+0000","last_fullsized":"2024-04-25T11:19:28.421617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818628+0000","last_change":"2024-04-25T11:19:27.457615+0000","last_active":"2024-04-25T11:19:28.818628+0000","last_peered":"2024-04-25T11:19:28.818628+0000","last_clean":"2024-04-25T11:19:28.818628+0000","last_became_active":"2024-04-25T11:19:27.447354+0000","last_became_peered":"2024-04-25T11:19:27.447354+0000","last_unstale":"2024-04-25T11:19:28.818628+0000","last_undegraded":"2024-04-25T11:19:28.818628+0000","last_fullsized":"2024-04-25T11:19:28.818628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.518039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.424873+0000","last_change":"2024-04-25T11:19:26.459930+0000","last_active":"2024-04-25T11:19:27.424873+0000","last_peered":"2024-04-25T11:19:27.424873+0000","last_clean":"2024-04-25T11:19:27.424873+0000","last_became_active":"2024-04-25T11:19:26.459292+0000","last_became_peered":"2024-04-25T11:19:26.459292+0000","last_unstale":"2024-04-25T11:19:27.424873+0000","last_undegraded":"2024-04-25T11:19:27.424873+0000","last_fullsized":"2024-04-25T11:19:27.424873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:08.607950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422340+0000","last_change":"2024-04-25T11:19:27.422008+0000","last_active":"2024-04-25T11:19:28.422340+0000","last_peered":"2024-04-25T11:19:28.422340+0000","last_clean":"2024-04-25T11:19:28.422340+0000","last_became_active":"2024-04-25T11:19:27.419929+0000","last_became_peered":"2024-04-25T11:19:27.419929+0000","last_unstale":"2024-04-25T11:19:28.422340+0000","last_undegraded":"2024-04-25T11:19:28.422340+0000","last_fullsized":"2024-04-25T11:19:28.422340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.431242+0000","last_change":"2024-04-25T11:19:26.435682+0000","last_active":"2024-04-25T11:19:28.431242+0000","last_peered":"2024-04-25T11:19:28.431242+0000","last_clean":"2024-04-25T11:19:28.431242+0000","last_became_active":"2024-04-25T11:19:26.435173+0000","last_became_peered":"2024-04-25T11:19:26.435173+0000","last_unstale":"2024-04-25T11:19:28.431242+0000","last_undegraded":"2024-04-25T11:19:28.431242+0000","last_fullsized":"2024-04-25T11:19:28.431242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:49.071318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451207+0000","last_change":"2024-04-25T11:19:27.446933+0000","last_active":"2024-04-25T11:19:27.451207+0000","last_peered":"2024-04-25T11:19:27.451207+0000","last_clean":"2024-04-25T11:19:27.451207+0000","last_became_active":"2024-04-25T11:19:27.446601+0000","last_became_peered":"2024-04-25T11:19:27.446601+0000","last_unstale":"2024-04-25T11:19:27.451207+0000","last_undegraded":"2024-04-25T11:19:27.451207+0000","last_fullsized":"2024-04-25T11:19:27.451207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.618925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433109+0000","last_change":"2024-04-25T11:19:26.451159+0000","last_active":"2024-04-25T11:19:28.433109+0000","last_peered":"2024-04-25T11:19:28.433109+0000","last_clean":"2024-04-25T11:19:28.433109+0000","last_became_active":"2024-04-25T11:19:26.450440+0000","last_became_peered":"2024-04-25T11:19:26.450440+0000","last_unstale":"2024-04-25T11:19:28.433109+0000","last_undegraded":"2024-04-25T11:19:28.433109+0000","last_fullsized":"2024-04-25T11:19:28.433109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.433525+0000","last_change":"2024-04-25T11:19:27.428326+0000","last_active":"2024-04-25T11:19:27.433525+0000","last_peered":"2024-04-25T11:19:27.433525+0000","last_clean":"2024-04-25T11:19:27.433525+0000","last_became_active":"2024-04-25T11:19:27.427906+0000","last_became_peered":"2024-04-25T11:19:27.427906+0000","last_unstale":"2024-04-25T11:19:27.433525+0000","last_undegraded":"2024-04-25T11:19:27.433525+0000","last_fullsized":"2024-04-25T11:19:27.433525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:25.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.405903+0000","last_change":"2024-04-25T11:19:26.447443+0000","last_active":"2024-04-25T11:19:27.405903+0000","last_peered":"2024-04-25T11:19:27.405903+0000","last_clean":"2024-04-25T11:19:27.405903+0000","last_became_active":"2024-04-25T11:19:26.439787+0000","last_became_peered":"2024-04-25T11:19:26.439787+0000","last_unstale":"2024-04-25T11:19:27.405903+0000","last_undegraded":"2024-04-25T11:19:27.405903+0000","last_fullsized":"2024-04-25T11:19:27.405903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:49.071318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.437428+0000","last_change":"2024-04-25T11:19:27.422934+0000","last_active":"2024-04-25T11:19:27.437428+0000","last_peered":"2024-04-25T11:19:27.437428+0000","last_clean":"2024-04-25T11:19:27.437428+0000","last_became_active":"2024-04-25T11:19:27.421387+0000","last_became_peered":"2024-04-25T11:19:27.421387+0000","last_unstale":"2024-04-25T11:19:27.437428+0000","last_undegraded":"2024-04-25T11:19:27.437428+0000","last_fullsized":"2024-04-25T11:19:27.437428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:47.993968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421663+0000","last_change":"2024-04-25T11:19:26.456701+0000","last_active":"2024-04-25T11:19:28.421663+0000","last_peered":"2024-04-25T11:19:28.421663+0000","last_clean":"2024-04-25T11:19:28.421663+0000","last_became_active":"2024-04-25T11:19:26.456252+0000","last_became_peered":"2024-04-25T11:19:26.456252+0000","last_unstale":"2024-04-25T11:19:28.421663+0000","last_undegraded":"2024-04-25T11:19:28.421663+0000","last_fullsized":"2024-04-25T11:19:28.421663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442721+0000","last_change":"2024-04-25T11:19:27.435245+0000","last_active":"2024-04-25T11:19:27.442721+0000","last_peered":"2024-04-25T11:19:27.442721+0000","last_clean":"2024-04-25T11:19:27.442721+0000","last_became_active":"2024-04-25T11:19:27.434609+0000","last_became_peered":"2024-04-25T11:19:27.434609+0000","last_unstale":"2024-04-25T11:19:27.442721+0000","last_undegraded":"2024-04-25T11:19:27.442721+0000","last_fullsized":"2024-04-25T11:19:27.442721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422471+0000","last_change":"2024-04-25T11:19:26.423167+0000","last_active":"2024-04-25T11:19:28.422471+0000","last_peered":"2024-04-25T11:19:28.422471+0000","last_clean":"2024-04-25T11:19:28.422471+0000","last_became_active":"2024-04-25T11:19:26.422397+0000","last_became_peered":"2024-04-25T11:19:26.422397+0000","last_unstale":"2024-04-25T11:19:28.422471+0000","last_undegraded":"2024-04-25T11:19:28.422471+0000","last_fullsized":"2024-04-25T11:19:28.422471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.048030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.432628+0000","last_change":"2024-04-25T11:19:27.426394+0000","last_active":"2024-04-25T11:19:27.432628+0000","last_peered":"2024-04-25T11:19:27.432628+0000","last_clean":"2024-04-25T11:19:27.432628+0000","last_became_active":"2024-04-25T11:19:27.424504+0000","last_became_peered":"2024-04-25T11:19:27.424504+0000","last_unstale":"2024-04-25T11:19:27.432628+0000","last_undegraded":"2024-04-25T11:19:27.432628+0000","last_fullsized":"2024-04-25T11:19:27.432628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.750362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.445666+0000","last_change":"2024-04-25T11:19:26.455784+0000","last_active":"2024-04-25T11:19:27.445666+0000","last_peered":"2024-04-25T11:19:27.445666+0000","last_clean":"2024-04-25T11:19:27.445666+0000","last_became_active":"2024-04-25T11:19:26.455125+0000","last_became_peered":"2024-04-25T11:19:26.455125+0000","last_unstale":"2024-04-25T11:19:27.445666+0000","last_undegraded":"2024-04-25T11:19:27.445666+0000","last_fullsized":"2024-04-25T11:19:27.445666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:40.075026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433197+0000","last_change":"2024-04-25T11:19:27.455131+0000","last_active":"2024-04-25T11:19:28.433197+0000","last_peered":"2024-04-25T11:19:28.433197+0000","last_clean":"2024-04-25T11:19:28.433197+0000","last_became_active":"2024-04-25T11:19:27.454732+0000","last_became_peered":"2024-04-25T11:19:27.454732+0000","last_unstale":"2024-04-25T11:19:28.433197+0000","last_undegraded":"2024-04-25T11:19:28.433197+0000","last_fullsized":"2024-04-25T11:19:28.433197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818302+0000","last_change":"2024-04-25T11:19:27.450900+0000","last_active":"2024-04-25T11:19:28.818302+0000","last_peered":"2024-04-25T11:19:28.818302+0000","last_clean":"2024-04-25T11:19:28.818302+0000","last_became_active":"2024-04-25T11:19:27.447933+0000","last_became_peered":"2024-04-25T11:19:27.447933+0000","last_unstale":"2024-04-25T11:19:28.818302+0000","last_undegraded":"2024-04-25T11:19:28.818302+0000","last_fullsized":"2024-04-25T11:19:28.818302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:03.621864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.461486+0000","last_change":"2024-04-25T11:19:26.456557+0000","last_active":"2024-04-25T11:19:27.461486+0000","last_peered":"2024-04-25T11:19:27.461486+0000","last_clean":"2024-04-25T11:19:27.461486+0000","last_became_active":"2024-04-25T11:19:26.455740+0000","last_became_peered":"2024-04-25T11:19:26.455740+0000","last_unstale":"2024-04-25T11:19:27.461486+0000","last_undegraded":"2024-04-25T11:19:27.461486+0000","last_fullsized":"2024-04-25T11:19:27.461486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:12.867967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818605+0000","last_change":"2024-04-25T11:19:27.440111+0000","last_active":"2024-04-25T11:19:28.818605+0000","last_peered":"2024-04-25T11:19:28.818605+0000","last_clean":"2024-04-25T11:19:28.818605+0000","last_became_active":"2024-04-25T11:19:27.430503+0000","last_became_peered":"2024-04-25T11:19:27.430503+0000","last_unstale":"2024-04-25T11:19:28.818605+0000","last_undegraded":"2024-04-25T11:19:28.818605+0000","last_fullsized":"2024-04-25T11:19:28.818605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:45.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.445898+0000","last_change":"2024-04-25T11:19:26.457866+0000","last_active":"2024-04-25T11:19:27.445898+0000","last_peered":"2024-04-25T11:19:27.445898+0000","last_clean":"2024-04-25T11:19:27.445898+0000","last_became_active":"2024-04-25T11:19:26.457223+0000","last_became_peered":"2024-04-25T11:19:26.457223+0000","last_unstale":"2024-04-25T11:19:27.445898+0000","last_undegraded":"2024-04-25T11:19:27.445898+0000","last_fullsized":"2024-04-25T11:19:27.445898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.048030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451829+0000","last_change":"2024-04-25T11:19:27.448305+0000","last_active":"2024-04-25T11:19:27.451829+0000","last_peered":"2024-04-25T11:19:27.451829+0000","last_clean":"2024-04-25T11:19:27.451829+0000","last_became_active":"2024-04-25T11:19:27.447940+0000","last_became_peered":"2024-04-25T11:19:27.447940+0000","last_unstale":"2024-04-25T11:19:27.451829+0000","last_undegraded":"2024-04-25T11:19:27.451829+0000","last_fullsized":"2024-04-25T11:19:27.451829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:04.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.448599+0000","last_change":"2024-04-25T11:19:26.425844+0000","last_active":"2024-04-25T11:19:27.448599+0000","last_peered":"2024-04-25T11:19:27.448599+0000","last_clean":"2024-04-25T11:19:27.448599+0000","last_became_active":"2024-04-25T11:19:26.425443+0000","last_became_peered":"2024-04-25T11:19:26.425443+0000","last_unstale":"2024-04-25T11:19:27.448599+0000","last_undegraded":"2024-04-25T11:19:27.448599+0000","last_fullsized":"2024-04-25T11:19:27.448599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.442448+0000","last_change":"2024-04-25T11:19:27.434918+0000","last_active":"2024-04-25T11:19:27.442448+0000","last_peered":"2024-04-25T11:19:27.442448+0000","last_clean":"2024-04-25T11:19:27.442448+0000","last_became_active":"2024-04-25T11:19:27.434444+0000","last_became_peered":"2024-04-25T11:19:27.434444+0000","last_unstale":"2024-04-25T11:19:27.442448+0000","last_undegraded":"2024-04-25T11:19:27.442448+0000","last_fullsized":"2024-04-25T11:19:27.442448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T11:19:27.451869+0000","last_change":"2024-04-25T11:19:26.448774+0000","last_active":"2024-04-25T11:19:27.451869+0000","last_peered":"2024-04-25T11:19:27.451869+0000","last_clean":"2024-04-25T11:19:27.451869+0000","last_became_active":"2024-04-25T11:19:26.448304+0000","last_became_peered":"2024-04-25T11:19:26.448304+0000","last_unstale":"2024-04-25T11:19:27.451869+0000","last_undegraded":"2024-04-25T11:19:27.451869+0000","last_fullsized":"2024-04-25T11:19:27.451869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:19:31.902 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:19:31.905 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:19:31.905 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:19:31.906 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T11:19:31.906 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T11:19:32.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176bcf2640 1 Processor -- start 2024-04-25T11:19:32.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176bcf2640 1 -- start start 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176bcf2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17640b5fc0 0x7f17641638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176bcf2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1764160210 con 0x7f176406dcb0 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176bcf2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176406c800 con 0x7f176406d8a0 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176bcf2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f176406d410 con 0x7f17640b5fc0 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176a28f640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f176406dcb0 0x7f17640af050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56982/0 (socket says 172.21.15.106:56982) 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176a28f640 1 -- 172.21.15.106:0/2684433573 learned_addr learned my addr 172.21.15.106:0/2684433573 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f176928d640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17640b5fc0 0x7f17641638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2360169274 0 0) 0x7f1764160210 con 0x7f176406dcb0 2024-04-25T11:19:32.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f174c003530 con 0x7f176406dcb0 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1534492161 0 0) 0x7f174c003530 con 0x7f176406dcb0 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17640b5fc0 msgr2=0x7f17641638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17640b5fc0 0x7f17641638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 >> v1:172.21.15.134:6789/0 conn(0x7f176406d8a0 legacy=0x7f17640b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176406cae0 con 0x7f176406dcb0 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f1754001680 con 0x7f176406dcb0 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.093+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 >> v1:172.21.15.106:6789/0 conn(0x7f176406dcb0 legacy=0x7f17640af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0039f0 0x7f174c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f174c004380 0x7f174c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f174c0050d0 0x7f174c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f176406d410 con 0x7f174c004380 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f174c00bf70 con 0x7f174c0050d0 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f174c00c170 con 0x7f174c0039f0 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176928d640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f174c004380 0x7f174c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1769a8e640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0039f0 0x7f174c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176928d640 1 -- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0039f0 msgr2=0x7f174c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176928d640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0039f0 0x7f174c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176928d640 1 -- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f174c0050d0 msgr2=0x7f174c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176928d640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f174c0050d0 0x7f174c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176928d640 1 -- 172.21.15.106:0/2684433573 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c003530 con 0x7f174c004380 2024-04-25T11:19:32.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1769a8e640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0039f0 0x7f174c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176928d640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f174c004380 0x7f174c0048d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1764073290 tx=0x7f175801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8e8e46a005df9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580b2070 con 0x7f174c004380 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17580a5c10 con 0x7f174c004380 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f1768a8c640 1 -- 172.21.15.106:0/2684433573 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758009c10 con 0x7f174c004380 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176bcf2640 1 -- 172.21.15.106:0/2684433573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f174c004380 msgr2=0x7f174c0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176bcf2640 1 --2- 172.21.15.106:0/2684433573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f174c004380 0x7f174c0048d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1764073290 tx=0x7f175801cc00 comp rx=0 tx=0).stop 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176a28f640 1 -- 172.21.15.106:0/2684433573 reap_dead start 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176bcf2640 1 -- 172.21.15.106:0/2684433573 shutdown_connections 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176bcf2640 1 -- 172.21.15.106:0/2684433573 >> 172.21.15.106:0/2684433573 conn(0x7f17640d1eb0 msgr2=0x7f17640d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176bcf2640 1 -- 172.21.15.106:0/2684433573 shutdown_connections 2024-04-25T11:19:32.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.097+0000 7f176bcf2640 1 -- 172.21.15.106:0/2684433573 wait complete. 2024-04-25T11:19:32.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 Processor -- start 2024-04-25T11:19:32.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 -- start start 2024-04-25T11:19:32.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c004380 0x7f17641f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1764162cf0 0x7f17640d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17640d1240 0x7f17641f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17640b5fc0 con 0x7f1764162cf0 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f176406d0a0 con 0x7f17640d1240 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17640b6140 con 0x7f174c004380 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1764162cf0 0x7f17640d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1764162cf0 0x7f17640d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46116/0 (socket says 172.21.15.106:46116) 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 -- 172.21.15.106:0/1654776567 learned_addr learned my addr 172.21.15.106:0/1654776567 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176a28f640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17640d1240 0x7f17641f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f1769a8e640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c004380 0x7f17641f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 -- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c004380 msgr2=0x7f17641f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c004380 0x7f17641f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 -- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17640d1240 msgr2=0x7f17641f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17640d1240 0x7f17641f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 -- 172.21.15.106:0/1654776567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17640d1650 con 0x7f1764162cf0 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176a28f640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17640d1240 0x7f17641f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176928d640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1764162cf0 0x7f17640d0d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f17580b0040 tx=0x7f1758009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f17627fc640 1 -- 172.21.15.106:0/1654776567 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580bc070 con 0x7f1764162cf0 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f17627fc640 1 -- 172.21.15.106:0/1654776567 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17580ada40 con 0x7f1764162cf0 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17640ce870 con 0x7f1764162cf0 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f17627fc640 1 -- 172.21.15.106:0/1654776567 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580b2040 con 0x7f1764162cf0 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.101+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17640b3090 con 0x7f1764162cf0 2024-04-25T11:19:32.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.105+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17640ced90 con 0x7f1764162cf0 2024-04-25T11:19:32.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.105+0000 7f17627fc640 1 -- 172.21.15.106:0/1654776567 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17580bf070 con 0x7f1764162cf0 2024-04-25T11:19:32.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.105+0000 7f17627fc640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f175006f430 0x7f1750071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.105+0000 7f17627fc640 1 -- 172.21.15.106:0/1654776567 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1758129750 con 0x7f1764162cf0 2024-04-25T11:19:32.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.105+0000 7f1769a8e640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f175006f430 0x7f1750071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.105+0000 7f1769a8e640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f175006f430 0x7f1750071940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f17640b4f90 tx=0x7f175c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.109+0000 7f17627fc640 1 -- 172.21.15.106:0/1654776567 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f17580c6020 con 0x7f1764162cf0 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.309+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f17640b1950 con 0x7f1764162cf0 2024-04-25T11:19:32.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.317+0000 7f17627fc640 1 -- 172.21.15.106:0/1654776567 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+179551 (secure 0 0 0) 0x7f17640d1650 con 0x7f1764162cf0 2024-04-25T11:19:32.318 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:32.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f175006f430 msgr2=0x7f1750071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f175006f430 0x7f1750071940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f17640b4f90 tx=0x7f175c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:32.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1764162cf0 msgr2=0x7f17640d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1764162cf0 0x7f17640d0d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f17580b0040 tx=0x7f1758009750 comp rx=0 tx=0).stop 2024-04-25T11:19:32.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 shutdown_connections 2024-04-25T11:19:32.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17640d1240 0x7f17641f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f175006f430 0x7f1750071940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1764162cf0 0x7f17640d0d00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 --2- 172.21.15.106:0/1654776567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c004380 0x7f17641f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 >> 172.21.15.106:0/1654776567 conn(0x7f17640d1eb0 msgr2=0x7f17640af220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 shutdown_connections 2024-04-25T11:19:32.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.321+0000 7f176bcf2640 1 -- 172.21.15.106:0/1654776567 wait complete. 2024-04-25T11:19:32.338 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":3760950393},{"type":"v1","addr":"172.21.15.106:6833","nonce":3760950393}]},"active_addr":"172.21.15.106:6833/3760950393","active_change":"2024-04-25T11:19:06.079838+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2303391505}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":994372073}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2031579350}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2945321224}]}]} 2024-04-25T11:19:32.342 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T11:19:32.342 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T11:19:32.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:32.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.481+0000 7f19fb136640 1 Processor -- start 2024-04-25T11:19:32.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.481+0000 7f19fb136640 1 -- start start 2024-04-25T11:19:32.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19fb136640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415e0a0 0x7f19f4073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19fb136640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f19f40d0ce0 0x7f19f40d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19fb136640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f40ce050 con 0x7f19f40ce850 2024-04-25T11:19:32.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19fb136640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f19f415e6b0 con 0x7f19f40d0ce0 2024-04-25T11:19:32.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19fb136640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f19f415e990 con 0x7f19f415e0a0 2024-04-25T11:19:32.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f19f40ce850 0x7f19f40cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56994/0 (socket says 172.21.15.106:56994) 2024-04-25T11:19:32.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f3fff640 1 -- 172.21.15.106:0/3609503026 learned_addr learned my addr 172.21.15.106:0/3609503026 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f8ed2640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415e0a0 0x7f19f4073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f96d3640 1 --2- 172.21.15.106:0/3609503026 >> v2:172.21.15.134:3300/0 conn(0x7f19f40d0ce0 0x7f19f40d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144156118 0 0) 0x7f19f40ce050 con 0x7f19f40ce850 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19dc003530 con 0x7f19f40ce850 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3957017143 0 0) 0x7f19dc003530 con 0x7f19f40ce850 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415e0a0 msgr2=0x7f19f4073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415e0a0 0x7f19f4073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 >> v2:172.21.15.134:3300/0 conn(0x7f19f40d0ce0 msgr2=0x7f19f40d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 --2- 172.21.15.106:0/3609503026 >> v2:172.21.15.134:3300/0 conn(0x7f19f40d0ce0 0x7f19f40d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f415ec70 con 0x7f19f40ce850 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f19e8001680 con 0x7f19f40ce850 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f96d3640 1 --2- 172.21.15.106:0/3609503026 >> v2:172.21.15.134:3300/0 conn(0x7f19f40d0ce0 0x7f19f40d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 >> v1:172.21.15.106:6789/0 conn(0x7f19f40ce850 legacy=0x7f19f40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415e0a0 0x7f19dc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc004d50 0x7f19dc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19f415e6b0 con 0x7f19dc003f40 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19f415e990 con 0x7f19dc004d50 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc00bad0 con 0x7f19f415e0a0 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f3fff640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f96d3640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc004d50 0x7f19dc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f3fff640 1 -- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415e0a0 msgr2=0x7f19dc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f8ed2640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415e0a0 0x7f19dc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f3fff640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415e0a0 0x7f19dc0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f3fff640 1 -- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc004d50 msgr2=0x7f19dc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f3fff640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc004d50 0x7f19dc005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.485+0000 7f19f3fff640 1 -- 172.21.15.106:0/3609503026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc003530 con 0x7f19dc003f40 2024-04-25T11:19:32.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19f3fff640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19dc004810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c270 tx=0x7f19e801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7a63c38870ad7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8007d40 con 0x7f19dc003f40 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8005b00 con 0x7f19dc003f40 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19f37fe640 1 -- 172.21.15.106:0/3609503026 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8007070 con 0x7f19dc003f40 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19fb136640 1 -- 172.21.15.106:0/3609503026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 msgr2=0x7f19dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19fb136640 1 --2- 172.21.15.106:0/3609503026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19dc004810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c270 tx=0x7f19e801f3f0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19f96d3640 1 -- 172.21.15.106:0/3609503026 reap_dead start 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19fb136640 1 -- 172.21.15.106:0/3609503026 shutdown_connections 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19fb136640 1 -- 172.21.15.106:0/3609503026 >> 172.21.15.106:0/3609503026 conn(0x7f19f406c7c0 msgr2=0x7f19f406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19fb136640 1 -- 172.21.15.106:0/3609503026 shutdown_connections 2024-04-25T11:19:32.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19fb136640 1 -- 172.21.15.106:0/3609503026 wait complete. 2024-04-25T11:19:32.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19fb136640 1 Processor -- start 2024-04-25T11:19:32.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.489+0000 7f19fb136640 1 -- start start 2024-04-25T11:19:32.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19f40b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415d310 0x7f19f41f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f415d720 0x7f19f41f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19f40d0860 con 0x7f19dc003f40 2024-04-25T11:19:32.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19f40d06e0 con 0x7f19f415d720 2024-04-25T11:19:32.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f19f415f270 con 0x7f19f415d310 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19f40b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19f40b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46134/0 (socket says 172.21.15.106:46134) 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 -- 172.21.15.106:0/4085063894 learned_addr learned my addr 172.21.15.106:0/4085063894 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f3fff640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415d310 0x7f19f41f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f96d3640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f415d720 0x7f19f41f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 -- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415d310 msgr2=0x7f19f41f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415d310 0x7f19f41f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 -- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f415d720 msgr2=0x7f19f41f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f415d720 0x7f19f41f7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 -- 172.21.15.106:0/4085063894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f40b05f0 con 0x7f19dc003f40 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f3fff640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415d310 0x7f19f41f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f8ed2640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19f40b00b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f19e4010890 tx=0x7f19e4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f17fa640 1 -- 172.21.15.106:0/4085063894 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e400f040 con 0x7f19dc003f40 2024-04-25T11:19:32.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f17fa640 1 -- 172.21.15.106:0/4085063894 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e400eb60 con 0x7f19dc003f40 2024-04-25T11:19:32.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f40ab970 con 0x7f19dc003f40 2024-04-25T11:19:32.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f17fa640 1 -- 172.21.15.106:0/4085063894 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e401c400 con 0x7f19dc003f40 2024-04-25T11:19:32.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f40abf30 con 0x7f19dc003f40 2024-04-25T11:19:32.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f40b3600 con 0x7f19dc003f40 2024-04-25T11:19:32.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f17fa640 1 -- 172.21.15.106:0/4085063894 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f19e401c5a0 con 0x7f19dc003f40 2024-04-25T11:19:32.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f17fa640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f19e006f430 0x7f19e0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f3fff640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f19e006f430 0x7f19e0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.493+0000 7f19f17fa640 1 -- 172.21.15.106:0/4085063894 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f19e408f8c0 con 0x7f19dc003f40 2024-04-25T11:19:32.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.497+0000 7f19f3fff640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f19e006f430 0x7f19e0071940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19e80ae000 tx=0x7f19f40cff70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.501+0000 7f19f17fa640 1 -- 172.21.15.106:0/4085063894 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f19f40b05f0 con 0x7f19dc003f40 2024-04-25T11:19:32.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.649+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19f40b3320 con 0x7f19dc003f40 2024-04-25T11:19:32.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.649+0000 7f19f17fa640 1 -- 172.21.15.106:0/4085063894 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f19e4018030 con 0x7f19dc003f40 2024-04-25T11:19:32.649 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:32.649 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:32.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f19e006f430 msgr2=0x7f19e0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f19e006f430 0x7f19e0071940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19e80ae000 tx=0x7f19f40cff70 comp rx=0 tx=0).stop 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 msgr2=0x7f19f40b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19f40b00b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f19e4010890 tx=0x7f19e4010d50 comp rx=0 tx=0).stop 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 shutdown_connections 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f415d720 0x7f19f41f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19f415d310 0x7f19f41f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f19e006f430 0x7f19e0071940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 --2- 172.21.15.106:0/4085063894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19dc003f40 0x7f19f40b00b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 >> 172.21.15.106:0/4085063894 conn(0x7f19f406c7c0 msgr2=0x7f19f40b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 shutdown_connections 2024-04-25T11:19:32.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.653+0000 7f19fb136640 1 -- 172.21.15.106:0/4085063894 wait complete. 2024-04-25T11:19:32.669 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T11:19:32.669 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:32.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.801+0000 7fefd8569640 1 Processor -- start 2024-04-25T11:19:32.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.801+0000 7fefd8569640 1 -- start start 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd8569640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fefd01424e0 0x7fefd00b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd8569640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd00b4fc0 0x7fefd0073900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd8569640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd0074250 con 0x7fefd0073e40 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd8569640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fefd00b53d0 con 0x7fefd01424e0 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd8569640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fefd015d700 con 0x7fefd00b4fc0 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd6305640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fefd01424e0 0x7fefd00b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd6305640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fefd01424e0 0x7fefd00b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46146/0 (socket says 172.21.15.106:46146) 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd6305640 1 -- 172.21.15.106:0/3805444040 learned_addr learned my addr 172.21.15.106:0/3805444040 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd5b04640 1 --2- 172.21.15.106:0/3805444040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd00b4fc0 0x7fefd0073900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd6305640 1 -- 172.21.15.106:0/3805444040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd00b4fc0 msgr2=0x7fefd0073900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd6305640 1 --2- 172.21.15.106:0/3805444040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd00b4fc0 0x7fefd0073900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd6305640 1 -- 172.21.15.106:0/3805444040 >> v1:172.21.15.134:6789/0 conn(0x7fefd0073e40 legacy=0x7fefd015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd6305640 1 -- 172.21.15.106:0/3805444040 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd0070660 con 0x7fefd01424e0 2024-04-25T11:19:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.805+0000 7fefd5b04640 1 --2- 172.21.15.106:0/3805444040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd00b4fc0 0x7fefd0073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd6305640 1 --2- 172.21.15.106:0/3805444040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd01424e0 0x7fefd00b4a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00bb10 tx=0x7fefcc01ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6488c3938b9eb281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd5303640 1 -- 172.21.15.106:0/3805444040 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0b4070 con 0x7fefd01424e0 2024-04-25T11:19:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd5303640 1 -- 172.21.15.106:0/3805444040 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc0a7910 con 0x7fefd01424e0 2024-04-25T11:19:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd5303640 1 -- 172.21.15.106:0/3805444040 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0a7ce0 con 0x7fefd01424e0 2024-04-25T11:19:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 -- 172.21.15.106:0/3805444040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd01424e0 msgr2=0x7fefd00b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 --2- 172.21.15.106:0/3805444040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd01424e0 0x7fefd00b4a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00bb10 tx=0x7fefcc01ea10 comp rx=0 tx=0).stop 2024-04-25T11:19:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 -- 172.21.15.106:0/3805444040 shutdown_connections 2024-04-25T11:19:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 --2- 172.21.15.106:0/3805444040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd00b4fc0 0x7fefd0073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 --2- 172.21.15.106:0/3805444040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd01424e0 0x7fefd00b4a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 -- 172.21.15.106:0/3805444040 >> 172.21.15.106:0/3805444040 conn(0x7fefd00b3780 msgr2=0x7fefd00b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 -- 172.21.15.106:0/3805444040 shutdown_connections 2024-04-25T11:19:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 -- 172.21.15.106:0/3805444040 wait complete. 2024-04-25T11:19:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 Processor -- start 2024-04-25T11:19:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.809+0000 7fefd8569640 1 -- start start 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd0073e40 0x7fefd014e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd00b4fc0 0x7fefd014f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefd01424e0 0x7fefd013f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fefd01428f0 con 0x7fefd00b4fc0 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefd015cae0 con 0x7fefd01424e0 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fefd015e3d0 con 0x7fefd0073e40 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd00b4fc0 0x7fefd014f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd00b4fc0 0x7fefd014f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46158/0 (socket says 172.21.15.106:46158) 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 -- 172.21.15.106:0/1824681975 learned_addr learned my addr 172.21.15.106:0/1824681975 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd6b06640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefd01424e0 0x7fefd013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd6305640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd0073e40 0x7fefd014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 -- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd0073e40 msgr2=0x7fefd014e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd0073e40 0x7fefd014e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 -- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefd01424e0 msgr2=0x7fefd013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefd01424e0 0x7fefd013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 -- 172.21.15.106:0/1824681975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd014d750 con 0x7fefd00b4fc0 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd6b06640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefd01424e0 0x7fefd013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd5b04640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd00b4fc0 0x7fefd014f050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fefc000c7f0 tx=0x7fefc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefc6ffd640 1 -- 172.21.15.106:0/1824681975 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0015070 con 0x7fefd00b4fc0 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefc6ffd640 1 -- 172.21.15.106:0/1824681975 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc00077c0 con 0x7fefd00b4fc0 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd014da30 con 0x7fefd00b4fc0 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefc6ffd640 1 -- 172.21.15.106:0/1824681975 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0007bb0 con 0x7fefd00b4fc0 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd014dff0 con 0x7fefd00b4fc0 2024-04-25T11:19:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd014c130 con 0x7fefd00b4fc0 2024-04-25T11:19:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.813+0000 7fefc6ffd640 1 -- 172.21.15.106:0/1824681975 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fefc0018070 con 0x7fefd00b4fc0 2024-04-25T11:19:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.817+0000 7fefc6ffd640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fefb406f400 0x7fefb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.817+0000 7fefd6305640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fefb406f400 0x7fefb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.817+0000 7fefc6ffd640 1 -- 172.21.15.106:0/1824681975 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fefc008f480 con 0x7fefd00b4fc0 2024-04-25T11:19:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.817+0000 7fefd6305640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fefb406f400 0x7fefb4071910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0b1000 tx=0x7fefcc0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.821+0000 7fefc6ffd640 1 -- 172.21.15.106:0/1824681975 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fefc005c370 con 0x7fefd00b4fc0 2024-04-25T11:19:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.969+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fefd0156d30 con 0x7fefd00b4fc0 2024-04-25T11:19:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.969+0000 7fefc6ffd640 1 -- 172.21.15.106:0/1824681975 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7fefc005bc70 con 0x7fefd00b4fc0 2024-04-25T11:19:32.970 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:32.970 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fefb406f400 msgr2=0x7fefb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fefb406f400 0x7fefb4071910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0b1000 tx=0x7fefcc0a9070 comp rx=0 tx=0).stop 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd00b4fc0 msgr2=0x7fefd014f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd00b4fc0 0x7fefd014f050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fefc000c7f0 tx=0x7fefc000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 shutdown_connections 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefd01424e0 0x7fefd013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fefb406f400 0x7fefb4071910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefd00b4fc0 0x7fefd014f050 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 --2- 172.21.15.106:0/1824681975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefd0073e40 0x7fefd014e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 >> 172.21.15.106:0/1824681975 conn(0x7fefd00b3780 msgr2=0x7fefd01552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 shutdown_connections 2024-04-25T11:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:32.973+0000 7fefd8569640 1 -- 172.21.15.106:0/1824681975 wait complete. 2024-04-25T11:19:32.990 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T11:19:32.990 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T11:19:32.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T11:19:32.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T11:19:32.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T11:19:32.992 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T11:19:32.992 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T11:19:32.992 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T11:19:33.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f3ab0809640 1 Processor -- start 2024-04-25T11:19:33.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f3ab0809640 1 -- start start 2024-04-25T11:19:33.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f83792dc640 1 Processor -- start 2024-04-25T11:19:33.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f83792dc640 1 -- start start 2024-04-25T11:19:33.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f3e58090640 1 Processor -- start 2024-04-25T11:19:33.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f3ab0809640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b5a00 0x7f3aac16ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f3ab0809640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aac09a530 con 0x7f3aac0b2de0 2024-04-25T11:19:33.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f3ab0809640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aac09a810 con 0x7f3aac164070 2024-04-25T11:19:33.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f3ab0809640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3aac09a2f0 con 0x7f3aac0b5a00 2024-04-25T11:19:33.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7fe5824d2640 1 Processor -- start 2024-04-25T11:19:33.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe5824d2640 1 -- start start 2024-04-25T11:19:33.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f83792dc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8374073cc0 0x7f83740740d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f83792dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 0x7f8374099be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f83792dc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83740c6920 con 0x7f837409a120 2024-04-25T11:19:33.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f83792dc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f837406e230 con 0x7f8374073cc0 2024-04-25T11:19:33.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f83792dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f83740b1600 con 0x7f8374074610 2024-04-25T11:19:33.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aa9d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3aac0b2de0 0x7f3aac0b31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57010/0 (socket says 172.21.15.106:57010) 2024-04-25T11:19:33.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aa9d9b640 1 -- 172.21.15.106:0/2385986659 learned_addr learned my addr 172.21.15.106:0/2385986659 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aaad9d640 1 --2- 172.21.15.106:0/2385986659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b5a00 0x7f3aac16ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f83737fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f837409a120 0x7f83740c70e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57020/0 (socket says 172.21.15.106:57020) 2024-04-25T11:19:33.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f83737fe640 1 -- 172.21.15.106:0/3953836531 learned_addr learned my addr 172.21.15.106:0/3953836531 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f8372ffd640 1 --2- 172.21.15.106:0/3953836531 >> v2:172.21.15.134:3300/0 conn(0x7f8374073cc0 0x7f83740740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aa959a640 1 -- 172.21.15.106:0/2385986659 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3786415069 0 0) 0x7f3aac09a530 con 0x7f3aac0b2de0 2024-04-25T11:19:33.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aa959a640 1 -- 172.21.15.106:0/2385986659 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a94003530 con 0x7f3aac0b2de0 2024-04-25T11:19:33.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f83727fc640 1 --2- 172.21.15.106:0/3953836531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 0x7f8374099be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aa959a640 1 -- 172.21.15.106:0/2385986659 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1435840540 0 0) 0x7f3aac09a810 con 0x7f3aac164070 2024-04-25T11:19:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aa959a640 1 -- 172.21.15.106:0/2385986659 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aac09a530 con 0x7f3aac164070 2024-04-25T11:19:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f8371ffb640 1 -- 172.21.15.106:0/3953836531 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1595120554 0 0) 0x7f83740c6920 con 0x7f837409a120 2024-04-25T11:19:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f8371ffb640 1 -- 172.21.15.106:0/3953836531 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f835c003530 con 0x7f837409a120 2024-04-25T11:19:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f8372ffd640 1 -- 172.21.15.106:0/3953836531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 msgr2=0x7f8374099be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f8372ffd640 1 --2- 172.21.15.106:0/3953836531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 0x7f8374099be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f8372ffd640 1 -- 172.21.15.106:0/3953836531 >> v1:172.21.15.106:6789/0 conn(0x7f837409a120 legacy=0x7f83740c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f8372ffd640 1 -- 172.21.15.106:0/3953836531 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83740b18e0 con 0x7f8374073cc0 2024-04-25T11:19:33.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aaad9d640 1 -- 172.21.15.106:0/2385986659 >> v1:172.21.15.134:6789/0 conn(0x7f3aac164070 legacy=0x7f3aac099be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aaad9d640 1 -- 172.21.15.106:0/2385986659 >> v1:172.21.15.106:6789/0 conn(0x7f3aac0b2de0 legacy=0x7f3aac0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3aaad9d640 1 -- 172.21.15.106:0/2385986659 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac09aaf0 con 0x7f3aac0b5a00 2024-04-25T11:19:33.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.209+0000 7f3e58090640 1 -- start start 2024-04-25T11:19:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe5824d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5740e38c0 0x7fe574052f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe5824d2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe574006c30 con 0x7fe5740384c0 2024-04-25T11:19:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe5824d2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe574006f10 con 0x7fe57404fc30 2024-04-25T11:19:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe5824d2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe5740071f0 con 0x7fe5740e38c0 2024-04-25T11:19:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e58090640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500d0cb0 0x7f3e501670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e58090640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e500b2c60 con 0x7f3e500cb7c0 2024-04-25T11:19:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e58090640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e500c6bb0 con 0x7f3e500cbbd0 2024-04-25T11:19:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e58090640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e500d0110 con 0x7f3e500d0cb0 2024-04-25T11:19:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e55e2c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3e500cb7c0 0x7f3e500b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57030/0 (socket says 172.21.15.106:57030) 2024-04-25T11:19:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e55e2c640 1 -- 172.21.15.106:0/4235216521 learned_addr learned my addr 172.21.15.106:0/4235216521 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e5662d640 1 --2- 172.21.15.106:0/4235216521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500d0cb0 0x7f3e501670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e54e2a640 1 -- 172.21.15.106:0/4235216521 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601990421 0 0) 0x7f3e500b2c60 con 0x7f3e500cb7c0 2024-04-25T11:19:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e54e2a640 1 -- 172.21.15.106:0/4235216521 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e3c007940 con 0x7f3e500cb7c0 2024-04-25T11:19:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe57bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5740e38c0 0x7fe574052f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe57bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5740e38c0 0x7fe574052f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46174/0 (socket says 172.21.15.106:46174) 2024-04-25T11:19:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe57bfff640 1 -- 172.21.15.106:0/2511287959 learned_addr learned my addr 172.21.15.106:0/2511287959 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f6abb59e640 1 Processor -- start 2024-04-25T11:19:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe57bfff640 1 -- 172.21.15.106:0/2511287959 >> v1:172.21.15.134:6789/0 conn(0x7fe57404fc30 legacy=0x7fe574050040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe57bfff640 1 -- 172.21.15.106:0/2511287959 >> v1:172.21.15.106:6789/0 conn(0x7fe5740384c0 legacy=0x7fe5740388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe57bfff640 1 -- 172.21.15.106:0/2511287959 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5740074d0 con 0x7fe5740e38c0 2024-04-25T11:19:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe78c74e640 1 Processor -- start 2024-04-25T11:19:33.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f6abb59e640 1 -- start start 2024-04-25T11:19:33.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7fe78c74e640 1 -- start start 2024-04-25T11:19:33.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e54e2a640 1 -- 172.21.15.106:0/4235216521 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1636066757 0 0) 0x7f3e3c007940 con 0x7f3e500cb7c0 2024-04-25T11:19:33.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e5662d640 1 -- 172.21.15.106:0/4235216521 >> v1:172.21.15.134:6789/0 conn(0x7f3e500cbbd0 legacy=0x7f3e500d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e5662d640 1 -- 172.21.15.106:0/4235216521 >> v1:172.21.15.106:6789/0 conn(0x7f3e500cb7c0 legacy=0x7f3e500b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.213+0000 7f3e5662d640 1 -- 172.21.15.106:0/4235216521 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e500cfac0 con 0x7f3e500d0cb0 2024-04-25T11:19:33.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3aaad9d640 1 --2- 172.21.15.106:0/2385986659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b5a00 0x7f3aac16ef30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0097f0 tx=0x7f3a9c01cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34cc1649b26708d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3aa959a640 1 -- 172.21.15.106:0/2385986659 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0ab5f0 con 0x7f3aac0b5a00 2024-04-25T11:19:33.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f8372ffd640 1 --2- 172.21.15.106:0/3953836531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374073cc0 0x7f83740740d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f836c009d90 tx=0x7f836c01c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11d576a36394581 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3aa959a640 1 -- 172.21.15.106:0/2385986659 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0b0070 con 0x7f3aac0b5a00 2024-04-25T11:19:33.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f8371ffb640 1 -- 172.21.15.106:0/3953836531 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0b2070 con 0x7f8374073cc0 2024-04-25T11:19:33.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3aa959a640 1 -- 172.21.15.106:0/2385986659 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0ab990 con 0x7f3aac0b5a00 2024-04-25T11:19:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abb59e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0ba850 0x7f6abc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abb59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6abc0d3170 0x7f6abc0d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abb59e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6abc0b2d20 con 0x7f6abc0ba440 2024-04-25T11:19:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abb59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6abc0b3610 con 0x7f6abc0d3170 2024-04-25T11:19:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abb59e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6abc0b9040 con 0x7f6abc0ba850 2024-04-25T11:19:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6abc0d3170 0x7f6abc0d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6abc0d3170 0x7f6abc0d3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46176/0 (socket says 172.21.15.106:46176) 2024-04-25T11:19:33.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abad9d640 1 -- 172.21.15.106:0/1868783586 learned_addr learned my addr 172.21.15.106:0/1868783586 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6ab9d9b640 1 --2- 172.21.15.106:0/1868783586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0ba850 0x7f6abc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f8371ffb640 1 -- 172.21.15.106:0/3953836531 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c0a58f0 con 0x7f8374073cc0 2024-04-25T11:19:33.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f8371ffb640 1 -- 172.21.15.106:0/3953836531 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0a5ca0 con 0x7f8374073cc0 2024-04-25T11:19:33.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa584eb7640 1 Processor -- start 2024-04-25T11:19:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3ab0809640 1 -- 172.21.15.106:0/2385986659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b5a00 msgr2=0x7f3aac16ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3ab0809640 1 --2- 172.21.15.106:0/2385986659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b5a00 0x7f3aac16ef30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0097f0 tx=0x7f3a9c01cf60 comp rx=0 tx=0).stop 2024-04-25T11:19:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3ab0809640 1 -- 172.21.15.106:0/2385986659 shutdown_connections 2024-04-25T11:19:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3ab0809640 1 --2- 172.21.15.106:0/2385986659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b5a00 0x7f3aac16ef30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3ab0809640 1 -- 172.21.15.106:0/2385986659 >> 172.21.15.106:0/2385986659 conn(0x7f3aac06cbf0 msgr2=0x7f3aac0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abad9d640 1 -- 172.21.15.106:0/1868783586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0ba850 msgr2=0x7f6abc0b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abad9d640 1 --2- 172.21.15.106:0/1868783586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0ba850 0x7f6abc0b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abad9d640 1 -- 172.21.15.106:0/1868783586 >> v1:172.21.15.134:6789/0 conn(0x7f6abc0ba440 legacy=0x7f6abc0b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f6abad9d640 1 -- 172.21.15.106:0/1868783586 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc0d47d0 con 0x7f6abc0d3170 2024-04-25T11:19:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f83792dc640 1 -- 172.21.15.106:0/3953836531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374073cc0 msgr2=0x7f83740740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f83792dc640 1 --2- 172.21.15.106:0/3953836531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374073cc0 0x7f83740740d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f836c009d90 tx=0x7f836c01c870 comp rx=0 tx=0).stop 2024-04-25T11:19:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa584eb7640 1 -- start start 2024-04-25T11:19:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f83792dc640 1 -- 172.21.15.106:0/3953836531 shutdown_connections 2024-04-25T11:19:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f83792dc640 1 --2- 172.21.15.106:0/3953836531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 0x7f8374099be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f83792dc640 1 --2- 172.21.15.106:0/3953836531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374073cc0 0x7f83740740d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f83792dc640 1 -- 172.21.15.106:0/3953836531 >> 172.21.15.106:0/3953836531 conn(0x7f83740cdc80 msgr2=0x7f83740ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3ab0809640 1 -- 172.21.15.106:0/2385986659 shutdown_connections 2024-04-25T11:19:33.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3ab0809640 1 -- 172.21.15.106:0/2385986659 wait complete. 2024-04-25T11:19:33.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f83792dc640 1 -- 172.21.15.106:0/3953836531 shutdown_connections 2024-04-25T11:19:33.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3e5662d640 1 --2- 172.21.15.106:0/4235216521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500d0cb0 0x7f3e501670f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00a890 tx=0x7f3e4c01c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bea9b495f484081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f83792dc640 1 -- 172.21.15.106:0/3953836531 wait complete. 2024-04-25T11:19:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa584eb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa580072d50 0x7fa580073160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3e54e2a640 1 -- 172.21.15.106:0/4235216521 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0b3070 con 0x7f3e500d0cb0 2024-04-25T11:19:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3e54e2a640 1 -- 172.21.15.106:0/4235216521 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e4c0ae480 con 0x7f3e500d0cb0 2024-04-25T11:19:33.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa584eb7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa5800d01d0 0x7fa5800d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa584eb7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5800736a0 con 0x7fa58006e8b0 2024-04-25T11:19:33.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa584eb7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa58006f3d0 con 0x7fa5800d01d0 2024-04-25T11:19:33.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa584eb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa58006f600 con 0x7fa580072d50 2024-04-25T11:19:33.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa57effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa58006e8b0 0x7fa58006ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57048/0 (socket says 172.21.15.106:57048) 2024-04-25T11:19:33.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fa57effd640 1 -- 172.21.15.106:0/1389459102 learned_addr learned my addr 172.21.15.106:0/1389459102 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe78c74e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78415fda0 0x7fe784164350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe78c74e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe7840b6160 0x7fe78406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe78c74e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe78406f3f0 con 0x7fe78406cd60 2024-04-25T11:19:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe78c74e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe7840b5cd0 con 0x7fe7840b6160 2024-04-25T11:19:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe78c74e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe784164890 con 0x7fe78415fda0 2024-04-25T11:19:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe78a4ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78415fda0 0x7fe784164350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe78a4ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78415fda0 0x7fe784164350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40076/0 (socket says 172.21.15.106:40076) 2024-04-25T11:19:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe78a4ea640 1 -- 172.21.15.106:0/1223843586 learned_addr learned my addr 172.21.15.106:0/1223843586 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe78a4ea640 1 -- 172.21.15.106:0/1223843586 >> v2:172.21.15.134:3300/0 conn(0x7fe7840b6160 msgr2=0x7fe78406c820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:19:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe78a4ea640 1 --2- 172.21.15.106:0/1223843586 >> v2:172.21.15.134:3300/0 conn(0x7fe7840b6160 0x7fe78406c820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe78a4ea640 1 -- 172.21.15.106:0/1223843586 >> v1:172.21.15.106:6789/0 conn(0x7fe78406cd60 legacy=0x7fe7840aefa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:19:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe78a4ea640 1 -- 172.21.15.106:0/1223843586 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78406d170 con 0x7fe78415fda0 2024-04-25T11:19:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57ffff640 1 --2- 172.21.15.106:0/1389459102 >> v2:172.21.15.134:3300/0 conn(0x7fa5800d01d0 0x7fa5800d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe57bfff640 1 --2- 172.21.15.106:0/2511287959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740e38c0 0x7fe574052f60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe570009870 tx=0x7fe57001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa166abac31c5db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe57affd640 1 -- 172.21.15.106:0/2511287959 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5700b0070 con 0x7fe5740e38c0 2024-04-25T11:19:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7fe57affd640 1 -- 172.21.15.106:0/2511287959 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5700ab600 con 0x7fe5740e38c0 2024-04-25T11:19:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.217+0000 7f3e54e2a640 1 -- 172.21.15.106:0/4235216521 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48001a00 con 0x7f3e500d0cb0 2024-04-25T11:19:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3e58090640 1 -- 172.21.15.106:0/4235216521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500d0cb0 msgr2=0x7f3e501670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3e58090640 1 --2- 172.21.15.106:0/4235216521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500d0cb0 0x7f3e501670f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00a890 tx=0x7f3e4c01c580 comp rx=0 tx=0).stop 2024-04-25T11:19:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe5824d2640 1 -- 172.21.15.106:0/2511287959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740e38c0 msgr2=0x7fe574052f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abad9d640 1 --2- 172.21.15.106:0/1868783586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0d3170 0x7f6abc0d3580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009870 tx=0x7f6ab001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8575eb056b5c1488 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3e58090640 1 -- 172.21.15.106:0/4235216521 shutdown_connections 2024-04-25T11:19:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3e58090640 1 --2- 172.21.15.106:0/4235216521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500d0cb0 0x7f3e501670f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3e58090640 1 -- 172.21.15.106:0/4235216521 >> 172.21.15.106:0/4235216521 conn(0x7f3e500cdc80 msgr2=0x7f3e500ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57f7fe640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa580072d50 0x7fa580073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3434717175 0 0) 0x7fa5700014a0 con 0x7fa58006e8b0 2024-04-25T11:19:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa56c003530 con 0x7fa58006e8b0 2024-04-25T11:19:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6ab959a640 1 -- 172.21.15.106:0/1868783586 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00b0070 con 0x7f6abc0d3170 2024-04-25T11:19:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6ab959a640 1 -- 172.21.15.106:0/1868783586 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab00ab600 con 0x7f6abc0d3170 2024-04-25T11:19:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6ab959a640 1 -- 172.21.15.106:0/1868783586 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00ab9d0 con 0x7f6abc0d3170 2024-04-25T11:19:33.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe57affd640 1 -- 172.21.15.106:0/2511287959 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5700b0070 con 0x7fe5740e38c0 2024-04-25T11:19:33.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe5824d2640 1 --2- 172.21.15.106:0/2511287959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740e38c0 0x7fe574052f60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe570009870 tx=0x7fe57001c490 comp rx=0 tx=0).stop 2024-04-25T11:19:33.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3ab0809640 1 Processor -- start 2024-04-25T11:19:33.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2172626234 0 0) 0x7fa56c003530 con 0x7fa58006e8b0 2024-04-25T11:19:33.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa580072d50 msgr2=0x7fa580073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa580072d50 0x7fa580073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 >> v2:172.21.15.134:3300/0 conn(0x7fa5800d01d0 msgr2=0x7fa5800d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 --2- 172.21.15.106:0/1389459102 >> v2:172.21.15.134:3300/0 conn(0x7fa5800d01d0 0x7fa5800d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58006f8e0 con 0x7fa58006e8b0 2024-04-25T11:19:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fa570001bd0 con 0x7fa58006e8b0 2024-04-25T11:19:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 >> v1:172.21.15.106:6789/0 conn(0x7fa58006e8b0 legacy=0x7fa58006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3e58090640 1 -- 172.21.15.106:0/4235216521 shutdown_connections 2024-04-25T11:19:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa56c0039c0 0x7fa56c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa580072d50 0x7fa56c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c004d50 0x7fa56c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa58006f600 con 0x7fa56c0039c0 2024-04-25T11:19:33.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c00bb60 con 0x7fa56c004d50 2024-04-25T11:19:33.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c00bd30 con 0x7fa580072d50 2024-04-25T11:19:33.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f83792dc640 1 Processor -- start 2024-04-25T11:19:33.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f83792dc640 1 -- start start 2024-04-25T11:19:33.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57ffff640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c004d50 0x7fa56c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3ab0809640 1 -- start start 2024-04-25T11:19:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abb59e640 1 -- 172.21.15.106:0/1868783586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0d3170 msgr2=0x7f6abc0d3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abb59e640 1 --2- 172.21.15.106:0/1868783586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0d3170 0x7f6abc0d3580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009870 tx=0x7f6ab001c440 comp rx=0 tx=0).stop 2024-04-25T11:19:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57f7fe640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa56c0039c0 0x7fa56c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe5824d2640 1 -- 172.21.15.106:0/2511287959 shutdown_connections 2024-04-25T11:19:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe5824d2640 1 --2- 172.21.15.106:0/2511287959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740e38c0 0x7fe574052f60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe5824d2640 1 -- 172.21.15.106:0/2511287959 >> 172.21.15.106:0/2511287959 conn(0x7fe574054d00 msgr2=0x7fe574055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe5824d2640 1 -- 172.21.15.106:0/2511287959 shutdown_connections 2024-04-25T11:19:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f3e58090640 1 -- 172.21.15.106:0/4235216521 wait complete. 2024-04-25T11:19:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abb59e640 1 -- 172.21.15.106:0/1868783586 shutdown_connections 2024-04-25T11:19:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abb59e640 1 --2- 172.21.15.106:0/1868783586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0d3170 0x7f6abc0d3580 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abb59e640 1 --2- 172.21.15.106:0/1868783586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0ba850 0x7f6abc0b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abb59e640 1 -- 172.21.15.106:0/1868783586 >> 172.21.15.106:0/1868783586 conn(0x7f6abc0d55a0 msgr2=0x7f6abc0d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abb59e640 1 -- 172.21.15.106:0/1868783586 shutdown_connections 2024-04-25T11:19:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57ffff640 1 -- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa580072d50 msgr2=0x7fa56c004810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:19:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57ffff640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa580072d50 0x7fa56c004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57ffff640 1 -- 172.21.15.106:0/1389459102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa56c0039c0 msgr2=0x7fa56c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57ffff640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa56c0039c0 0x7fa56c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57ffff640 1 -- 172.21.15.106:0/1389459102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58006f8e0 con 0x7fa56c004d50 2024-04-25T11:19:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fa57f7fe640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa56c0039c0 0x7fa56c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7f6abb59e640 1 -- 172.21.15.106:0/1868783586 wait complete. 2024-04-25T11:19:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.221+0000 7fe5824d2640 1 -- 172.21.15.106:0/2511287959 wait complete. 2024-04-25T11:19:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83792dc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374073cc0 0x7f83740b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3e58090640 1 Processor -- start 2024-04-25T11:19:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3e58090640 1 -- start start 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83792dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 0x7f83741595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83792dc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f837409a120 0x7f837415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f8372ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374073cc0 0x7f83740b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83737fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f837409a120 0x7f837415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83737fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f837409a120 0x7f837415fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46180/0 (socket says 172.21.15.106:46180) 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83737fe640 1 -- 172.21.15.106:0/4059444588 learned_addr learned my addr 172.21.15.106:0/4059444588 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83727fc640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 0x7f83741595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8374074a20 con 0x7f837409a120 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83740c6bc0 con 0x7f8374073cc0 2024-04-25T11:19:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f837409a940 con 0x7f8374074610 2024-04-25T11:19:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b2de0 0x7f3aac0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3aac0b5a00 0x7f3aac0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac164070 0x7f3aac15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac172020 con 0x7f3aac0b5a00 2024-04-25T11:19:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac171ea0 con 0x7f3aac164070 2024-04-25T11:19:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3aac1721a0 con 0x7f3aac0b2de0 2024-04-25T11:19:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3aac0b5a00 0x7f3aac0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3aac0b5a00 0x7f3aac0cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46196/0 (socket says 172.21.15.106:46196) 2024-04-25T11:19:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 -- 172.21.15.106:0/2061433793 learned_addr learned my addr 172.21.15.106:0/2061433793 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aaad9d640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac164070 0x7f3aac15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fe5824d2640 1 Processor -- start 2024-04-25T11:19:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 -- 172.21.15.106:0/2061433793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b2de0 msgr2=0x7f3aac0cc1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:19:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac0b2de0 0x7f3aac0cc1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 -- 172.21.15.106:0/2061433793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac164070 msgr2=0x7f3aac15da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac164070 0x7f3aac15da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 -- 172.21.15.106:0/2061433793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac15df90 con 0x7f3aac0b5a00 2024-04-25T11:19:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fe5824d2640 1 -- start start 2024-04-25T11:19:33.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fa57ffff640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c004d50 0x7fa56c005160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa56800ba00 tx=0x7fa56801c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7764fdd731186982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fe78a4ea640 1 --2- 172.21.15.106:0/1223843586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78415fda0 0x7fe784164350 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe780013150 tx=0x7fe780024c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c3a84abfafbcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f6abb59e640 1 Processor -- start 2024-04-25T11:19:33.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3aa9d9b640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3aac0b5a00 0x7f3aac0cdaa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400cbc0 tx=0x7f3aa400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83727fc640 1 -- 172.21.15.106:0/4059444588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374073cc0 msgr2=0x7f83740b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83727fc640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374073cc0 0x7f83740b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83727fc640 1 -- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f837409a120 msgr2=0x7f837415fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83727fc640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f837409a120 0x7f837415fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83727fc640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83740af080 con 0x7f8374074610 2024-04-25T11:19:33.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83737fe640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f837409a120 0x7f837415fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:33.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56800b600 con 0x7fa56c004d50 2024-04-25T11:19:33.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5680b1070 con 0x7fa56c004d50 2024-04-25T11:19:33.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fa57e7fc640 1 -- 172.21.15.106:0/1389459102 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5680ac680 con 0x7fa56c004d50 2024-04-25T11:19:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4017070 con 0x7f3aac0b5a00 2024-04-25T11:19:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa400ad40 con 0x7f3aac0b5a00 2024-04-25T11:19:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4012600 con 0x7f3aac0b5a00 2024-04-25T11:19:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aac0cca00 con 0x7f3aac0b5a00 2024-04-25T11:19:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aac0ccfc0 con 0x7f3aac0b5a00 2024-04-25T11:19:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3e58090640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500cb7c0 0x7f3e501fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3e58090640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e500cbbd0 0x7f3e501fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3e58090640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e500d0cb0 0x7f3e501fe320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f83727fc640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 0x7f83741595f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f836800e870 tx=0x7f836800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3aac050750 con 0x7f3aac0b5a00 2024-04-25T11:19:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fe5824d2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740384c0 0x7fe5740d1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f3e55e2c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500cb7c0 0x7f3e501fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e55e2c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500cb7c0 0x7f3e501fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40112/0 (socket says 172.21.15.106:40112) 2024-04-25T11:19:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe5824d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57404fc30 0x7fe5740ddd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e55e2c640 1 -- 172.21.15.106:0/3445152544 learned_addr learned my addr 172.21.15.106:0/3445152544 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e5562b640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e500cbbd0 0x7f3e501fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe5824d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740e38c0 0x7fe57404d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe5824d2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740f1f20 con 0x7fe57404fc30 2024-04-25T11:19:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e5662d640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e500d0cb0 0x7f3e501fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe5824d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740f1da0 con 0x7fe5740e38c0 2024-04-25T11:19:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe5824d2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe5740f20a0 con 0x7fe5740384c0 2024-04-25T11:19:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e58090640 1 -- 172.21.15.106:0/3445152544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5006e0e0 con 0x7f3e500d0cb0 2024-04-25T11:19:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368017070 con 0x7f8374074610 2024-04-25T11:19:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e58090640 1 -- 172.21.15.106:0/3445152544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e500c6840 con 0x7f3e500cbbd0 2024-04-25T11:19:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e58090640 1 -- 172.21.15.106:0/3445152544 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5006e260 con 0x7f3e500cb7c0 2024-04-25T11:19:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83740af360 con 0x7f8374074610 2024-04-25T11:19:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740384c0 0x7fe5740d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740384c0 0x7fe5740d1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40126/0 (socket says 172.21.15.106:40126) 2024-04-25T11:19:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 -- 172.21.15.106:0/3926994207 learned_addr learned my addr 172.21.15.106:0/3926994207 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836800ee60 con 0x7f8374074610 2024-04-25T11:19:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836800cb60 con 0x7f8374074610 2024-04-25T11:19:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe580a6f640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740e38c0 0x7fe57404d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7f6abb59e640 1 -- start start 2024-04-25T11:19:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f6abb59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6abc0ba440 0x7f6abc154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe7894e8640 1 -- 172.21.15.106:0/1223843586 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800adae0 con 0x7fe78415fda0 2024-04-25T11:19:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe7894e8640 1 -- 172.21.15.106:0/1223843586 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7800adc80 con 0x7fe78415fda0 2024-04-25T11:19:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe7894e8640 1 -- 172.21.15.106:0/1223843586 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800112f0 con 0x7fe78415fda0 2024-04-25T11:19:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe78c74e640 1 -- 172.21.15.106:0/1223843586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78415fda0 msgr2=0x7fe784164350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe78c74e640 1 --2- 172.21.15.106:0/1223843586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78415fda0 0x7fe784164350 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe780013150 tx=0x7fe780024c50 comp rx=0 tx=0).stop 2024-04-25T11:19:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe78c74e640 1 -- 172.21.15.106:0/1223843586 shutdown_connections 2024-04-25T11:19:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe78c74e640 1 --2- 172.21.15.106:0/1223843586 >> v2:172.21.15.134:3300/0 conn(0x7fe7840b6160 0x7fe78406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe78c74e640 1 --2- 172.21.15.106:0/1223843586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78415fda0 0x7fe784164350 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe78c74e640 1 -- 172.21.15.106:0/1223843586 >> 172.21.15.106:0/1223843586 conn(0x7fe7840b3560 msgr2=0x7fe7840b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe78c74e640 1 -- 172.21.15.106:0/1223843586 shutdown_connections 2024-04-25T11:19:33.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe78c74e640 1 -- 172.21.15.106:0/1223843586 wait complete. 2024-04-25T11:19:33.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83740af920 con 0x7f8374074610 2024-04-25T11:19:33.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8374159b30 con 0x7f8374074610 2024-04-25T11:19:33.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fa584eb7640 1 -- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c004d50 msgr2=0x7fa56c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fa584eb7640 1 --2- 172.21.15.106:0/1389459102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c004d50 0x7fa56c005160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa56800ba00 tx=0x7fa56801c6a0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.225+0000 7fa57ffff640 1 -- 172.21.15.106:0/1389459102 reap_dead start 2024-04-25T11:19:33.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 -- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740e38c0 msgr2=0x7fe57404d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5740e38c0 0x7fe57404d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 -- 172.21.15.106:0/3926994207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57404fc30 msgr2=0x7fe5740ddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57404fc30 0x7fe5740ddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 -- 172.21.15.106:0/3926994207 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5740d1e70 con 0x7fe5740384c0 2024-04-25T11:19:33.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e5662d640 1 -- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500cb7c0 msgr2=0x7f3e501fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e5662d640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e500cb7c0 0x7f3e501fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e5662d640 1 -- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e500cbbd0 msgr2=0x7f3e501fdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e5662d640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e500cbbd0 0x7f3e501fdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e5662d640 1 -- 172.21.15.106:0/3445152544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e50202050 con 0x7f3e500d0cb0 2024-04-25T11:19:33.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e5662d640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e500d0cb0 0x7f3e501fe320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e500721b0 tx=0x7f3e4c00a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e467fc640 1 -- 172.21.15.106:0/3445152544 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0b3040 con 0x7f3e500d0cb0 2024-04-25T11:19:33.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3e58090640 1 -- 172.21.15.106:0/3445152544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e50201290 con 0x7f3e500d0cb0 2024-04-25T11:19:33.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa15e4b2640 1 Processor -- start 2024-04-25T11:19:33.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa15e4b2640 1 -- start start 2024-04-25T11:19:33.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa15e4b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d1fa0 0x7fa1580d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa15e4b2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1580d17b0 0x7fa1580b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa15e4b2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1580c6cf0 con 0x7fa1580d23b0 2024-04-25T11:19:33.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa15e4b2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa1580c6aa0 con 0x7fa1580d17b0 2024-04-25T11:19:33.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa15e4b2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa1580b8490 con 0x7fa1580d1fa0 2024-04-25T11:19:33.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe57bfff640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740384c0 0x7fe5740d1930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe5700ae040 tx=0x7fe570019230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5700b0040 con 0x7fe5740384c0 2024-04-25T11:19:33.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5700ab400 con 0x7fe5740384c0 2024-04-25T11:19:33.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740397f0 con 0x7fe5740384c0 2024-04-25T11:19:33.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5740edfa0 con 0x7fe5740384c0 2024-04-25T11:19:33.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa1577fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fa1580d23b0 0x7fa1580b6260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:56460/0 (socket says 172.21.15.106:56460) 2024-04-25T11:19:33.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa1577fe640 1 -- 172.21.15.106:0/895160325 learned_addr learned my addr 172.21.15.106:0/895160325 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fa157fff640 1 --2- 172.21.15.106:0/895160325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d1fa0 0x7fa1580d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5740ec610 con 0x7fe5740384c0 2024-04-25T11:19:33.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f6abb59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0ba850 0x7f6abc1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f6aba59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6abc0ba440 0x7f6abc154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f6aba59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6abc0ba440 0x7f6abc154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45202/0 (socket says 172.21.15.106:45202) 2024-04-25T11:19:33.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3aa401a070 con 0x7f3aac0b5a00 2024-04-25T11:19:33.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3a9affd640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a8c06f4d0 0x7f3a8c0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3aa408fb80 con 0x7f3aac0b5a00 2024-04-25T11:19:33.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3a9affd640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7f3a8c0777a0 0x7f3a8c079bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 --> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a8c07a2c0 con 0x7f3a8c0777a0 2024-04-25T11:19:33.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.229+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3aa408ff70 con 0x7f3aac0b5a00 2024-04-25T11:19:33.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa584eb7640 1 -- 172.21.15.106:0/1389459102 shutdown_connections 2024-04-25T11:19:33.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa584eb7640 1 -- 172.21.15.106:0/1389459102 >> 172.21.15.106:0/1389459102 conn(0x7fa5800b3770 msgr2=0x7fa5800d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f3aaad9d640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7f3a8c0777a0 0x7f3a8c079bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa584eb7640 1 -- 172.21.15.106:0/1389459102 shutdown_connections 2024-04-25T11:19:33.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa157fff640 1 -- 172.21.15.106:0/895160325 >> v2:172.21.15.106:3300/0 conn(0x7fa1580d17b0 msgr2=0x7fa1580b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa157fff640 1 --2- 172.21.15.106:0/895160325 >> v2:172.21.15.106:3300/0 conn(0x7fa1580d17b0 0x7fa1580b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa157fff640 1 -- 172.21.15.106:0/895160325 >> v1:172.21.15.134:6789/0 conn(0x7fa1580d23b0 legacy=0x7fa1580b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa157fff640 1 -- 172.21.15.106:0/895160325 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1580b8be0 con 0x7fa1580d1fa0 2024-04-25T11:19:33.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f3aaad9d640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7f3a8c0777a0 0x7f3a8c079bf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f3aaa59c640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a8c06f4d0 0x7f3a8c0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe574005580 con 0x7fe5740384c0 2024-04-25T11:19:33.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f3e58090640 1 -- 172.21.15.106:0/3445152544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e50201850 con 0x7f3e500d0cb0 2024-04-25T11:19:33.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f3aaa59c640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a8c06f4d0 0x7f3a8c0719e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00014e0 tx=0x7f3aa00078b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6ab9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0ba850 0x7f6abc1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6ab9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0ba850 0x7f6abc1555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46218/0 (socket says 172.21.15.106:46218) 2024-04-25T11:19:33.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6ab9d9b640 1 -- 172.21.15.106:0/2889447021 learned_addr learned my addr 172.21.15.106:0/2889447021 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6abb59e640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0d3170 0x7f6abc156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6abb59e640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc0d4450 con 0x7f6abc0ba850 2024-04-25T11:19:33.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6abb59e640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc0d42d0 con 0x7f6abc0ba440 2024-04-25T11:19:33.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6abb59e640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6abc16ad20 con 0x7f6abc0d3170 2024-04-25T11:19:33.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f3e467fc640 1 -- 172.21.15.106:0/3445152544 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e4c004230 con 0x7f3e500d0cb0 2024-04-25T11:19:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6abad9d640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0d3170 0x7f6abc156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aba59c640 1 -- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0d3170 msgr2=0x7f6abc156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aba59c640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6abc0d3170 0x7f6abc156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aba59c640 1 -- 172.21.15.106:0/2889447021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0ba850 msgr2=0x7f6abc1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aba59c640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6abc0ba850 0x7f6abc1555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aba59c640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc0a29e0 con 0x7f6abc0ba440 2024-04-25T11:19:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f836801a070 con 0x7f8374074610 2024-04-25T11:19:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f835b7fe640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f836006f430 0x7f8360071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f836808f810 con 0x7f8374074610 2024-04-25T11:19:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f835b7fe640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7f8360077680 0x7f8360079ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8368016ec0 con 0x7f8360077680 2024-04-25T11:19:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f836808fc70 con 0x7f8374074610 2024-04-25T11:19:33.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fe78c74e640 1 Processor -- start 2024-04-25T11:19:33.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fe78c74e640 1 -- start start 2024-04-25T11:19:33.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aba59c640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6abc0ba440 0x7f6abc154c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00c3c0 tx=0x7f6aac00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f8372ffd640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f836006f430 0x7f8360071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa157fff640 1 --2- 172.21.15.106:0/895160325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d1fa0 0x7fa1580d1270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0097a0 tx=0x7fa14c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cfe5377063a64ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe5700bd030 con 0x7fe5740384c0 2024-04-25T11:19:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac017070 con 0x7f6abc0ba440 2024-04-25T11:19:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aac0052d0 con 0x7f6abc0ba440 2024-04-25T11:19:33.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac003670 con 0x7f6abc0ba440 2024-04-25T11:19:33.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f83737fe640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7f8360077680 0x7f8360079ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f8372ffd640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f836006f430 0x7f8360071940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f836c0af000 tx=0x7f836c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 <== osd.2 v2:172.21.15.106:6808/3117805672 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a8c07a2c0 con 0x7f3a8c0777a0 2024-04-25T11:19:33.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa584eb7640 1 -- 172.21.15.106:0/1389459102 wait complete. 2024-04-25T11:19:33.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fa584eb7640 1 Processor -- start 2024-04-25T11:19:33.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fe578ff9640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe56006f400 0x7fe560071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7f6abb59e640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc0a1940 con 0x7f6abc0ba440 2024-04-25T11:19:33.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.233+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe570129f80 con 0x7fe5740384c0 2024-04-25T11:19:33.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f6abb59e640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc0a1f00 con 0x7f6abc0ba440 2024-04-25T11:19:33.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe57b7fe640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe56006f400 0x7fe560071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe578ff9640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fe560077650 0x7fe560079aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 --> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe56007a170 con 0x7fe560077650 2024-04-25T11:19:33.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe57012a410 con 0x7fe5740384c0 2024-04-25T11:19:33.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f83737fe640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7f8360077680 0x7f8360079ad0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f3e467fc640 1 -- 172.21.15.106:0/3445152544 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0ae960 con 0x7f3e500d0cb0 2024-04-25T11:19:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f3e467fc640 1 -- 172.21.15.106:0/3445152544 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3e4c0c4060 con 0x7f3e500d0cb0 2024-04-25T11:19:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe580a6f640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fe560077650 0x7fe560079aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa156ffd640 1 -- 172.21.15.106:0/895160325 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0ab070 con 0x7fa1580d1fa0 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78c74e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78406cd60 0x7fe7841569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78c74e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7840b6160 0x7fe78414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78c74e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe78415fda0 0x7fe7841f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78406cd60 0x7fe7841569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78406cd60 0x7fe7841569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40156/0 (socket says 172.21.15.106:40156) 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 -- 172.21.15.106:0/1202164869 learned_addr learned my addr 172.21.15.106:0/1202164869 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7840afac0 con 0x7fe78415fda0 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7840b5960 con 0x7fe7840b6160 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe7841f6e10 con 0x7fe78406cd60 2024-04-25T11:19:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe789ce9640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7840b6160 0x7fe78414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 -- 172.21.15.106:0/1202164869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7840b6160 msgr2=0x7fe78414beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7840b6160 0x7fe78414beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 -- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe78415fda0 msgr2=0x7fe7841f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe78415fda0 0x7fe7841f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7840b1170 con 0x7fe78406cd60 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78a4ea640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78406cd60 0x7fe7841569a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe780013150 tx=0x7fe7800219f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800110d0 con 0x7fe78406cd60 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe780011270 con 0x7fe78406cd60 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780011660 con 0x7fe78406cd60 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe784157310 con 0x7fe78406cd60 2024-04-25T11:19:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7841578d0 con 0x7fe78406cd60 2024-04-25T11:19:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 <== osd.1 v2:172.21.15.106:6800/3323217021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8368016ec0 con 0x7f8360077680 2024-04-25T11:19:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa584eb7640 1 -- start start 2024-04-25T11:19:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa584eb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa56c004d50 0x7fa580208b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa584eb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58015f640 0x7fa5800af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa584eb7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58015fa50 0x7fa5800af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa584eb7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5800d0ca0 con 0x7fa58015fa50 2024-04-25T11:19:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa584eb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5800d0b20 con 0x7fa58015f640 2024-04-25T11:19:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa584eb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5800b4d40 con 0x7fa56c004d50 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa56c004d50 0x7fa580208b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa56c004d50 0x7fa580208b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40160/0 (socket says 172.21.15.106:40160) 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57f7fe640 1 -- 172.21.15.106:0/2430678746 learned_addr learned my addr 172.21.15.106:0/2430678746 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57effd640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58015f640 0x7fa5800af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57f7fe640 1 -- 172.21.15.106:0/2430678746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58015f640 msgr2=0x7fa5800af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57f7fe640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58015f640 0x7fa5800af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57f7fe640 1 -- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58015fa50 msgr2=0x7fa5800af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57f7fe640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58015fa50 0x7fa5800af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa57f7fe640 1 -- 172.21.15.106:0/2430678746 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58009bf80 con 0x7fa56c004d50 2024-04-25T11:19:33.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fe57b7fe640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe56006f400 0x7fe560071910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe568000f70 tx=0x7fe5680079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f3e467fc640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3e3406f500 0x7f3e34071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f3e467fc640 1 -- 172.21.15.106:0/3445152544 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3e4c12acb0 con 0x7f3e500d0cb0 2024-04-25T11:19:33.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f3e3bfff640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f3e30001660 0x7f3e30003b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 --> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e30006c10 con 0x7f3e30001660 2024-04-25T11:19:33.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f3e5562b640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f3e30001660 0x7f3e30003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f3e5562b640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f3e30001660 0x7f3e30003b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa156ffd640 1 -- 172.21.15.106:0/895160325 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14c0a5610 con 0x7fa1580d1fa0 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa156ffd640 1 -- 172.21.15.106:0/895160325 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0a59e0 con 0x7fa1580d1fa0 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa15e4b2640 1 -- 172.21.15.106:0/895160325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d1fa0 msgr2=0x7fa1580d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa15e4b2640 1 --2- 172.21.15.106:0/895160325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d1fa0 0x7fa1580d1270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0097a0 tx=0x7fa14c01c000 comp rx=0 tx=0).stop 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa15e4b2640 1 -- 172.21.15.106:0/895160325 shutdown_connections 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa15e4b2640 1 --2- 172.21.15.106:0/895160325 >> v2:172.21.15.106:3300/0 conn(0x7fa1580d17b0 0x7fa1580b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa15e4b2640 1 --2- 172.21.15.106:0/895160325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d1fa0 0x7fa1580d1270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa15e4b2640 1 -- 172.21.15.106:0/895160325 >> 172.21.15.106:0/895160325 conn(0x7fa1580cfe90 msgr2=0x7fa1580d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa15e4b2640 1 -- 172.21.15.106:0/895160325 shutdown_connections 2024-04-25T11:19:33.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7fa15e4b2640 1 -- 172.21.15.106:0/895160325 wait complete. 2024-04-25T11:19:33.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.237+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a94001040 con 0x7f6abc0ba440 2024-04-25T11:19:33.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa57f7fe640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa56c004d50 0x7fa580208b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa578011590 tx=0x7fa578011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe784050750 con 0x7fe78406cd60 2024-04-25T11:19:33.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578015870 con 0x7fa56c004d50 2024-04-25T11:19:33.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa584eb7640 1 -- 172.21.15.106:0/2430678746 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58015ffb0 con 0x7fa56c004d50 2024-04-25T11:19:33.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa584eb7640 1 -- 172.21.15.106:0/2430678746 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58009ba10 con 0x7fa56c004d50 2024-04-25T11:19:33.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa578015a10 con 0x7fa56c004d50 2024-04-25T11:19:33.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578015dc0 con 0x7fa56c004d50 2024-04-25T11:19:33.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7f3e467fc640 1 -- 172.21.15.106:0/3445152544 <== osd.5 v2:172.21.15.134:6800/1995022176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e30006c10 con 0x7f3e30001660 2024-04-25T11:19:33.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa15e4b2640 1 Processor -- start 2024-04-25T11:19:33.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa15e4b2640 1 -- start start 2024-04-25T11:19:33.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa584eb7640 1 -- 172.21.15.106:0/2430678746 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa564001040 con 0x7fa56c004d50 2024-04-25T11:19:33.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7f3e55e2c640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3e3406f500 0x7f3e34071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fe580a6f640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fe560077650 0x7fe560079aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7f3e55e2c640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3e3406f500 0x7f3e34071a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e48001340 tx=0x7f3e48007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa57801d400 con 0x7fa56c004d50 2024-04-25T11:19:33.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 <== osd.4 v2:172.21.15.134:6808/4246182191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe56007a170 con 0x7fe560077650 2024-04-25T11:19:33.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fa577fff640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa56006f400 0x7fa560071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa578090090 con 0x7fa56c004d50 2024-04-25T11:19:33.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa577fff640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7fa560077650 0x7fa560079aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 --> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5780110b0 con 0x7fa560077650 2024-04-25T11:19:33.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa578090520 con 0x7fa56c004d50 2024-04-25T11:19:33.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15e4b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1580d17b0 0x7fa1581674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15e4b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1580d1fa0 0x7fa158175040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15e4b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d23b0 0x7fa158175580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15e4b2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa158169af0 con 0x7fa1580d1fa0 2024-04-25T11:19:33.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa157fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1580d17b0 0x7fa1581674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa57effd640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa56006f400 0x7fa560071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa57ffff640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7fa560077650 0x7fa560079aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15e4b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa158169970 con 0x7fa1580d17b0 2024-04-25T11:19:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa157fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1580d17b0 0x7fa1581674e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45240/0 (socket says 172.21.15.106:45240) 2024-04-25T11:19:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa157fff640 1 -- 172.21.15.106:0/2741531340 learned_addr learned my addr 172.21.15.106:0/2741531340 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15e4b2640 1 -- 172.21.15.106:0/2741531340 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa158169c70 con 0x7fa1580d23b0 2024-04-25T11:19:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15ca4f640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d23b0 0x7fa158175580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa57effd640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa56006f400 0x7fa560071910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa58009a5d0 tx=0x7fa570001cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa57ffff640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7fa560077650 0x7fa560079aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15ca4f640 1 -- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1580d17b0 msgr2=0x7fa1581674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15ca4f640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1580d17b0 0x7fa1581674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15ca4f640 1 -- 172.21.15.106:0/2741531340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1580d1fa0 msgr2=0x7fa158175040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15ca4f640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1580d1fa0 0x7fa158175040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15ca4f640 1 -- 172.21.15.106:0/2741531340 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1580ca5c0 con 0x7fa1580d23b0 2024-04-25T11:19:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15ca4f640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d23b0 0x7fa158175580 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa15000be50 tx=0x7fa15000bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150019070 con 0x7fa1580d23b0 2024-04-25T11:19:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa150014440 con 0x7fa1580d23b0 2024-04-25T11:19:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150014830 con 0x7fa1580d23b0 2024-04-25T11:19:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15e4b2640 1 -- 172.21.15.106:0/2741531340 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1580c6fb0 con 0x7fa1580d23b0 2024-04-25T11:19:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 <== osd.3 v2:172.21.15.106:6824/3914961038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5780110b0 con 0x7fa560077650 2024-04-25T11:19:33.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa15e4b2640 1 -- 172.21.15.106:0/2741531340 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1580c7570 con 0x7fa1580d23b0 2024-04-25T11:19:33.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa130001040 con 0x7fa1580d23b0 2024-04-25T11:19:33.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7800c9020 con 0x7fe78406cd60 2024-04-25T11:19:33.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.241+0000 7fe77affd640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe76806f430 0x7fe768071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe780132ac0 con 0x7fe78406cd60 2024-04-25T11:19:33.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fe77affd640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7fe7680776a0 0x7fe768079af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe780010e90 con 0x7fe7680776a0 2024-04-25T11:19:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.245+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe780132eb0 con 0x7fe78406cd60 2024-04-25T11:19:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fe789ce9640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe76806f430 0x7fe768071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fe789ce9640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe76806f430 0x7fe768071940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7840b3e70 tx=0x7fe774009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa15001c070 con 0x7fa1580d23b0 2024-04-25T11:19:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fa154ff9640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa13c06f400 0x7fa13c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa15008f960 con 0x7fa1580d23b0 2024-04-25T11:19:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fa154ff9640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7fa13c0775f0 0x7fa13c079a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 --> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa150010e50 con 0x7fa13c0775f0 2024-04-25T11:19:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa15008fdf0 con 0x7fa1580d23b0 2024-04-25T11:19:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6aac01a070 con 0x7f6abc0ba440 2024-04-25T11:19:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6aaaffd640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a9c06f430 0x7f6a9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6aac08e890 con 0x7f6abc0ba440 2024-04-25T11:19:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6aaaffd640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f6a9c077680 0x7f6a9c079ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6aac01ee50 con 0x7f6a9c077680 2024-04-25T11:19:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6aac08ec40 con 0x7f6abc0ba440 2024-04-25T11:19:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6ab9d9b640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a9c06f430 0x7f6a9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fa1577fe640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7fa13c0775f0 0x7fa13c079a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6abad9d640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f6a9c077680 0x7f6a9c079ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6ab9d9b640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a9c06f430 0x7f6a9c071940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6abc15a870 tx=0x7f6ab40098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7fa1577fe640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7fa13c0775f0 0x7fa13c079a40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.249+0000 7f6abad9d640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f6a9c077680 0x7f6a9c079ad0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.253+0000 7fe78aceb640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7fe7680776a0 0x7fe768079af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.253+0000 7fa157fff640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa13c06f400 0x7fa13c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.253+0000 7fa157fff640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa13c06f400 0x7fa13c071910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0097a0 tx=0x7fa14c0b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.253+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 <== osd.6 v2:172.21.15.134:6824/2059200670 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa150010e50 con 0x7fa13c0775f0 2024-04-25T11:19:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.253+0000 7fe78aceb640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7fe7680776a0 0x7fe768079af0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.253+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 <== osd.7 v2:172.21.15.134:6816/1198352866 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6aac01ee50 con 0x7f6a9c077680 2024-04-25T11:19:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.253+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 <== osd.0 v2:172.21.15.106:6809/70365189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe780010e90 con 0x7fe7680776a0 2024-04-25T11:19:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.257+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 --> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3aac0b29b0 con 0x7f3a8c0777a0 2024-04-25T11:19:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.257+0000 7f3a9affd640 1 -- 172.21.15.106:0/2061433793 <== osd.2 v2:172.21.15.106:6808/3117805672 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3aac0b29b0 con 0x7f3a8c0777a0 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7f3a8c0777a0 msgr2=0x7f3a8c079bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7f3a8c0777a0 0x7f3a8c079bf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a8c06f4d0 msgr2=0x7f3a8c0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a8c06f4d0 0x7f3a8c0719e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00014e0 tx=0x7f3aa00078b0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3aac0b5a00 msgr2=0x7f3aac0cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 --2- 172.21.15.106:0/2061433793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3aac0b5a00 0x7f3aac0cdaa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400cbc0 tx=0x7f3aa400a430 comp rx=0 tx=0).stop 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3aaad9d640 1 -- 172.21.15.106:0/2061433793 reap_dead start 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 shutdown_connections 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 >> 172.21.15.106:0/2061433793 conn(0x7f3aac06cbf0 msgr2=0x7f3aac0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 shutdown_connections 2024-04-25T11:19:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.261+0000 7f3ab0809640 1 -- 172.21.15.106:0/2061433793 wait complete. 2024-04-25T11:19:33.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 --> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3e30005cf0 con 0x7f3e30001660 2024-04-25T11:19:33.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e467fc640 1 -- 172.21.15.106:0/3445152544 <== osd.5 v2:172.21.15.134:6800/1995022176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3e30005cf0 con 0x7f3e30001660 2024-04-25T11:19:33.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f3e30001660 msgr2=0x7f3e30003b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e3bfff640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f3e30001660 0x7f3e30003b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7fa584eb7640 1 -- 172.21.15.106:0/2430678746 --> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa564002d60 con 0x7fa560077650 2024-04-25T11:19:33.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3e3406f500 msgr2=0x7f3e34071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e3bfff640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3e3406f500 0x7f3e34071a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e48001340 tx=0x7f3e48007dd0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e500d0cb0 msgr2=0x7f3e501fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e3bfff640 1 --2- 172.21.15.106:0/3445152544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e500d0cb0 0x7f3e501fe320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e500721b0 tx=0x7f3e4c00a1a0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7f3e5662d640 1 -- 172.21.15.106:0/3445152544 reap_dead start 2024-04-25T11:19:33.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.269+0000 7fa577fff640 1 -- 172.21.15.106:0/2430678746 <== osd.3 v2:172.21.15.106:6824/3914961038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa564002d60 con 0x7fa560077650 2024-04-25T11:19:33.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 shutdown_connections 2024-04-25T11:19:33.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 --> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83740aee10 con 0x7f8360077680 2024-04-25T11:19:33.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 >> 172.21.15.106:0/3445152544 conn(0x7f3e500cdc80 msgr2=0x7f3e5006ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 --> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe57404f130 con 0x7fe560077650 2024-04-25T11:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 shutdown_connections 2024-04-25T11:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 -- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7fa560077650 msgr2=0x7fa560079aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7fa560077650 0x7fa560079aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f3e3bfff640 1 -- 172.21.15.106:0/3445152544 wait complete. 2024-04-25T11:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 -- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa56006f400 msgr2=0x7fa560071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa56006f400 0x7fa560071910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa58009a5d0 tx=0x7fa570001cb0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f835b7fe640 1 -- 172.21.15.106:0/4059444588 <== osd.1 v2:172.21.15.106:6800/3323217021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83740aee10 con 0x7f8360077680 2024-04-25T11:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 -- 172.21.15.106:0/2430678746 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa56c004d50 msgr2=0x7fa580208b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 --2- 172.21.15.106:0/2430678746 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa56c004d50 0x7fa580208b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa578011590 tx=0x7fa578011a50 comp rx=0 tx=0).stop 2024-04-25T11:19:33.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa57ffff640 1 -- 172.21.15.106:0/2430678746 reap_dead start 2024-04-25T11:19:33.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 -- 172.21.15.106:0/2430678746 shutdown_connections 2024-04-25T11:19:33.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 -- 172.21.15.106:0/2430678746 >> 172.21.15.106:0/2430678746 conn(0x7fa5800b3770 msgr2=0x7fa580075800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 -- 172.21.15.106:0/2430678746 shutdown_connections 2024-04-25T11:19:33.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fa575ffb640 1 -- 172.21.15.106:0/2430678746 wait complete. 2024-04-25T11:19:33.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe578ff9640 1 -- 172.21.15.106:0/3926994207 <== osd.4 v2:172.21.15.134:6808/4246182191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe57404f130 con 0x7fe560077650 2024-04-25T11:19:33.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7f8360077680 msgr2=0x7f8360079ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7f8360077680 0x7f8360079ad0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f836006f430 msgr2=0x7f8360071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f836006f430 0x7f8360071940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f836c0af000 tx=0x7f836c0a7070 comp rx=0 tx=0).stop 2024-04-25T11:19:33.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 msgr2=0x7f83741595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 --2- 172.21.15.106:0/4059444588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8374074610 0x7f83741595f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f836800e870 tx=0x7f836800ed30 comp rx=0 tx=0).stop 2024-04-25T11:19:33.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83737fe640 1 -- 172.21.15.106:0/4059444588 reap_dead start 2024-04-25T11:19:33.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 shutdown_connections 2024-04-25T11:19:33.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 >> 172.21.15.106:0/4059444588 conn(0x7f83740cdc80 msgr2=0x7f837406fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 shutdown_connections 2024-04-25T11:19:33.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7f83792dc640 1 -- 172.21.15.106:0/4059444588 wait complete. 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fe560077650 msgr2=0x7fe560079aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fe560077650 0x7fe560079aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe56006f400 msgr2=0x7fe560071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe56006f400 0x7fe560071910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe568000f70 tx=0x7fe5680079c0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740384c0 msgr2=0x7fe5740d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 --2- 172.21.15.106:0/3926994207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5740384c0 0x7fe5740d1930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe5700ae040 tx=0x7fe570019230 comp rx=0 tx=0).stop 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe580a6f640 1 -- 172.21.15.106:0/3926994207 reap_dead start 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 shutdown_connections 2024-04-25T11:19:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.273+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 >> 172.21.15.106:0/3926994207 conn(0x7fe574054d00 msgr2=0x7fe5740e2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.277+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 shutdown_connections 2024-04-25T11:19:33.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 --> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6a94002de0 con 0x7f6a9c077680 2024-04-25T11:19:33.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7fe5824d2640 1 -- 172.21.15.106:0/3926994207 wait complete. 2024-04-25T11:19:33.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 --> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa130002de0 con 0x7fa13c0775f0 2024-04-25T11:19:33.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aaaffd640 1 -- 172.21.15.106:0/2889447021 <== osd.7 v2:172.21.15.134:6816/1198352866 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6a94002de0 con 0x7f6a9c077680 2024-04-25T11:19:33.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f6a9c077680 msgr2=0x7f6a9c079ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f6a9c077680 0x7f6a9c079ad0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7fa154ff9640 1 -- 172.21.15.106:0/2741531340 <== osd.6 v2:172.21.15.134:6824/2059200670 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa130002de0 con 0x7fa13c0775f0 2024-04-25T11:19:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a9c06f430 msgr2=0x7f6a9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a9c06f430 0x7f6a9c071940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6abc15a870 tx=0x7f6ab40098d0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6abc0ba440 msgr2=0x7f6abc154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 --2- 172.21.15.106:0/2889447021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6abc0ba440 0x7f6abc154c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00c3c0 tx=0x7f6aac00c880 comp rx=0 tx=0).stop 2024-04-25T11:19:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6abad9d640 1 -- 172.21.15.106:0/2889447021 reap_dead start 2024-04-25T11:19:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 shutdown_connections 2024-04-25T11:19:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 >> 172.21.15.106:0/2889447021 conn(0x7f6abc0d55a0 msgr2=0x7f6abc071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 shutdown_connections 2024-04-25T11:19:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.281+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/2889447021 wait complete. 2024-04-25T11:19:33.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7fa13c0775f0 msgr2=0x7fa13c079a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7fa13c0775f0 0x7fa13c079a40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa13c06f400 msgr2=0x7fa13c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa13c06f400 0x7fa13c071910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0097a0 tx=0x7fa14c0b2040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d23b0 msgr2=0x7fa158175580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 --2- 172.21.15.106:0/2741531340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1580d23b0 0x7fa158175580 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa15000be50 tx=0x7fa15000bf40 comp rx=0 tx=0).stop 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa15ca4f640 1 -- 172.21.15.106:0/2741531340 reap_dead start 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 shutdown_connections 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 >> 172.21.15.106:0/2741531340 conn(0x7fa1580cfe90 msgr2=0x7fa1580b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 shutdown_connections 2024-04-25T11:19:33.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.285+0000 7fa1427fc640 1 -- 172.21.15.106:0/2741531340 wait complete. 2024-04-25T11:19:33.327 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T11:19:33.328 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:19:33.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.293+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 --> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe784050750 con 0x7fe7680776a0 2024-04-25T11:19:33.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.293+0000 7fe77affd640 1 -- 172.21.15.106:0/1202164869 <== osd.0 v2:172.21.15.106:6809/70365189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe784050750 con 0x7fe7680776a0 2024-04-25T11:19:33.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.293+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7fe7680776a0 msgr2=0x7fe768079af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.293+0000 7fe78c74e640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7fe7680776a0 0x7fe768079af0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe76806f430 msgr2=0x7fe768071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78c74e640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe76806f430 0x7fe768071940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7840b3e70 tx=0x7fe774009060 comp rx=0 tx=0).stop 2024-04-25T11:19:33.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78406cd60 msgr2=0x7fe7841569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78c74e640 1 --2- 172.21.15.106:0/1202164869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe78406cd60 0x7fe7841569a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe780013150 tx=0x7fe7800219f0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78aceb640 1 -- 172.21.15.106:0/1202164869 reap_dead start 2024-04-25T11:19:33.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 shutdown_connections 2024-04-25T11:19:33.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 >> 172.21.15.106:0/1202164869 conn(0x7fe7840b3560 msgr2=0x7fe7840b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 shutdown_connections 2024-04-25T11:19:33.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.297+0000 7fe78c74e640 1 -- 172.21.15.106:0/1202164869 wait complete. 2024-04-25T11:19:33.331 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T11:19:33.332 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:19:33.332 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:33.333 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T11:19:33.334 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:33.334 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:19:33.335 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T11:19:33.335 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T11:19:33.336 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:33.336 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T11:19:33.337 INFO:teuthology.orchestra.run.smithi106.stdout:55834574857 2024-04-25T11:19:33.337 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T11:19:33.338 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:33.338 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:19:33.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.589+0000 7f24ccefc640 1 Processor -- start 2024-04-25T11:19:33.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.589+0000 7f24ccefc640 1 -- start start 2024-04-25T11:19:33.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.589+0000 7fda7fc69640 1 Processor -- start 2024-04-25T11:19:33.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.589+0000 7fda7fc69640 1 -- start start 2024-04-25T11:19:33.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.589+0000 7fb35202e640 1 Processor -- start 2024-04-25T11:19:33.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24ccefc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f24c806e8b0 0x7f24c806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24ccefc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c80d0cf0 con 0x7f24c80d01d0 2024-04-25T11:19:33.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24ccefc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c806f600 con 0x7f24c8072d50 2024-04-25T11:19:33.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24ccefc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f24c806f8e0 con 0x7f24c806e8b0 2024-04-25T11:19:33.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.589+0000 7fb35202e640 1 -- start start 2024-04-25T11:19:33.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf894a8640 1 Processor -- start 2024-04-25T11:19:33.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f24c80d01d0 0x7f24c80d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57058/0 (socket says 172.21.15.106:57058) 2024-04-25T11:19:33.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c7fff640 1 -- 172.21.15.106:0/1954887067 learned_addr learned my addr 172.21.15.106:0/1954887067 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.589+0000 7f44ac881640 1 Processor -- start 2024-04-25T11:19:33.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.589+0000 7f44ac881640 1 -- start start 2024-04-25T11:19:33.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44ac881640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80700a0 0x7f44a815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44ac881640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44a80ce140 0x7f44a80ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44ac881640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a815c130 con 0x7f44a815ce10 2024-04-25T11:19:33.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44ac881640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f44a815c440 con 0x7f44a80ce140 2024-04-25T11:19:33.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44ac881640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f44a80cea90 con 0x7f44a80700a0 2024-04-25T11:19:33.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44a759e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44a80ce140 0x7f44a80ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44a759e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44a80ce140 0x7f44a80ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46260/0 (socket says 172.21.15.106:46260) 2024-04-25T11:19:33.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44a759e640 1 -- 172.21.15.106:0/1385778370 learned_addr learned my addr 172.21.15.106:0/1385778370 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf894a8640 1 -- start start 2024-04-25T11:19:33.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7fc69640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700eb920 0x7fda700e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7fc69640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda70033ab0 0x7fda70033ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7fc69640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda70038380 con 0x7fda700539c0 2024-04-25T11:19:33.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7fc69640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fda700060d0 con 0x7fda70033ab0 2024-04-25T11:19:33.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7fc69640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fda700063b0 con 0x7fda700eb920 2024-04-25T11:19:33.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c6ffd640 1 --2- 172.21.15.106:0/1954887067 >> v2:172.21.15.134:3300/0 conn(0x7f24c806e8b0 0x7f24c806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7ff4d2f8b640 1 Processor -- start 2024-04-25T11:19:33.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44a6d9d640 1 --2- 172.21.15.106:0/1385778370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80700a0 0x7f44a815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1633406624 0 0) 0x7f24c80d0cf0 con 0x7f24c80d01d0 2024-04-25T11:19:33.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24b0003530 con 0x7f24c80d01d0 2024-04-25T11:19:33.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35202e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb34c06e8b0 0x7fb34c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35202e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb34c0d01d0 0x7fb34c0d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35202e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34c06f600 con 0x7fb34c072d50 2024-04-25T11:19:33.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35202e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb34c06f8e0 con 0x7fb34c06e8b0 2024-04-25T11:19:33.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35202e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb34c06fbc0 con 0x7fb34c0d01d0 2024-04-25T11:19:33.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7ff4d2f8b640 1 -- start start 2024-04-25T11:19:33.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f43c2a89640 1 Processor -- start 2024-04-25T11:19:33.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f43c2a89640 1 -- start start 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2031800067 0 0) 0x7f24b0003530 con 0x7f24c80d01d0 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 >> v2:172.21.15.134:3300/0 conn(0x7f24c806e8b0 msgr2=0x7f24c806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 --2- 172.21.15.106:0/1954887067 >> v2:172.21.15.134:3300/0 conn(0x7f24c806e8b0 0x7f24c806ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 >> v1:172.21.15.134:6789/0 conn(0x7f24c8072d50 legacy=0x7f24c8073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c806fbc0 con 0x7f24c80d01d0 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f24bc001680 con 0x7f24c80d01d0 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 >> v1:172.21.15.106:6789/0 conn(0x7f24c80d01d0 legacy=0x7f24c80d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b00039c0 0x7f24b0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24b0004810 0x7f24b0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24b0005360 0x7f24b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24c806f8e0 con 0x7f24b00039c0 2024-04-25T11:19:33.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24b000c290 con 0x7f24b0005360 2024-04-25T11:19:33.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f24b000c490 con 0x7f24b0004810 2024-04-25T11:19:33.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c6ffd640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24b0004810 0x7f24b0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35182d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb34c0d01d0 0x7fb34c0d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35182d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb34c0d01d0 0x7fb34c0d05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45262/0 (socket says 172.21.15.106:45262) 2024-04-25T11:19:33.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35182d640 1 -- 172.21.15.106:0/3488638985 learned_addr learned my addr 172.21.15.106:0/3488638985 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35082b640 1 --2- 172.21.15.106:0/3488638985 >> v2:172.21.15.106:3300/0 conn(0x7fb34c06e8b0 0x7fb34c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7da05640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700eb920 0x7fda700e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7e206640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fda700539c0 0x7fda70053dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:56474/0 (socket says 172.21.15.106:56474) 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7e206640 1 -- 172.21.15.106:0/3532399095 learned_addr learned my addr 172.21.15.106:0/3532399095 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7d204640 1 --2- 172.21.15.106:0/3532399095 >> v2:172.21.15.106:3300/0 conn(0x7fda70033ab0 0x7fda70033ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7ca03640 1 -- 172.21.15.106:0/3532399095 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997669835 0 0) 0x7fda78050cc0 con 0x7fda700539c0 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7ca03640 1 -- 172.21.15.106:0/3532399095 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda64003530 con 0x7fda700539c0 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44a759e640 1 -- 172.21.15.106:0/1385778370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80700a0 msgr2=0x7f44a815c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44a759e640 1 --2- 172.21.15.106:0/1385778370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80700a0 0x7f44a815c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44a759e640 1 -- 172.21.15.106:0/1385778370 >> v1:172.21.15.134:6789/0 conn(0x7f44a815ce10 legacy=0x7f44a80b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f44a759e640 1 -- 172.21.15.106:0/1385778370 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a80b5a90 con 0x7f44a80ce140 2024-04-25T11:19:33.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35182d640 1 -- 172.21.15.106:0/3488638985 >> v2:172.21.15.106:3300/0 conn(0x7fb34c06e8b0 msgr2=0x7fb34c06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb33ffff640 1 -- 172.21.15.106:0/3488638985 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515353177 0 0) 0x7fb34c06f600 con 0x7fb34c072d50 2024-04-25T11:19:33.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c77fe640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b00039c0 0x7f24b0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35182d640 1 --2- 172.21.15.106:0/3488638985 >> v2:172.21.15.106:3300/0 conn(0x7fb34c06e8b0 0x7fb34c06ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35182d640 1 -- 172.21.15.106:0/3488638985 >> v1:172.21.15.134:6789/0 conn(0x7fb34c072d50 legacy=0x7fb34c073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35182d640 1 -- 172.21.15.106:0/3488638985 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c06fea0 con 0x7fb34c0d01d0 2024-04-25T11:19:33.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf894a8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcf8415f2d0 0x7fcf84152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf894a8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf8415d4d0 0x7fcf8415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f106f937640 1 Processor -- start 2024-04-25T11:19:33.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf894a8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8415e230 con 0x7fcf84159e30 2024-04-25T11:19:33.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf894a8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8415e510 con 0x7fcf8415f2d0 2024-04-25T11:19:33.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf894a8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8415e7f0 con 0x7fcf8415d4d0 2024-04-25T11:19:33.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fb35082b640 1 --2- 172.21.15.106:0/3488638985 >> v2:172.21.15.106:3300/0 conn(0x7fb34c06e8b0 0x7fb34c06ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:33.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf837fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf8415d4d0 0x7fcf8415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf837fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf8415d4d0 0x7fcf8415d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45270/0 (socket says 172.21.15.106:45270) 2024-04-25T11:19:33.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fcf837fe640 1 -- 172.21.15.106:0/528802820 learned_addr learned my addr 172.21.15.106:0/528802820 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c6ffd640 1 -- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24b0005360 msgr2=0x7f24b000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c6ffd640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24b0005360 0x7f24b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c6ffd640 1 -- 172.21.15.106:0/1954887067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b00039c0 msgr2=0x7f24b0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c6ffd640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b00039c0 0x7f24b0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7f24c6ffd640 1 -- 172.21.15.106:0/1954887067 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c806fbc0 con 0x7f24b0004810 2024-04-25T11:19:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7ff4d2f8b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff4cc06d600 0x7ff4cc0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7ff4d2f8b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4cc06de50 con 0x7ff4cc0d29b0 2024-04-25T11:19:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7ff4d2f8b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4cc0afec0 con 0x7ff4cc15cbb0 2024-04-25T11:19:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7ff4d2f8b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff4cc0afc90 con 0x7ff4cc06d600 2024-04-25T11:19:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7ff4cbfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff4cc0d29b0 0x7ff4cc0b09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57064/0 (socket says 172.21.15.106:57064) 2024-04-25T11:19:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7ff4cbfff640 1 -- 172.21.15.106:0/2972572698 learned_addr learned my addr 172.21.15.106:0/2972572698 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f24c7fff640 1 -- 172.21.15.106:0/1954887067 reap_dead start 2024-04-25T11:19:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7ff4d1528640 1 --2- 172.21.15.106:0/2972572698 >> v2:172.21.15.134:3300/0 conn(0x7ff4cc06d600 0x7ff4cc0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fcf83fff640 1 --2- 172.21.15.106:0/528802820 >> v2:172.21.15.106:3300/0 conn(0x7fcf8415f2d0 0x7fcf84152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fcf837fe640 1 -- 172.21.15.106:0/528802820 >> v2:172.21.15.106:3300/0 conn(0x7fcf8415f2d0 msgr2=0x7fcf84152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fcf837fe640 1 --2- 172.21.15.106:0/528802820 >> v2:172.21.15.106:3300/0 conn(0x7fcf8415f2d0 0x7fcf84152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fcf837fe640 1 -- 172.21.15.106:0/528802820 >> v1:172.21.15.134:6789/0 conn(0x7fcf84159e30 legacy=0x7fcf8415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fcf837fe640 1 -- 172.21.15.106:0/528802820 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8415ead0 con 0x7fcf8415d4d0 2024-04-25T11:19:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7ff4d1528640 1 -- 172.21.15.106:0/2972572698 >> v1:172.21.15.134:6789/0 conn(0x7ff4cc15cbb0 legacy=0x7ff4cc0d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7ff4d1528640 1 -- 172.21.15.106:0/2972572698 >> v1:172.21.15.106:6789/0 conn(0x7ff4cc0d29b0 legacy=0x7ff4cc0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7ff4d1528640 1 -- 172.21.15.106:0/2972572698 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc1635d0 con 0x7ff4cc06d600 2024-04-25T11:19:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44a759e640 1 --2- 172.21.15.106:0/1385778370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44a80ce140 0x7f44a80ce550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009bd0 tx=0x7f44a001cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ef28f9f91b877b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7d204640 1 -- 172.21.15.106:0/3532399095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700eb920 msgr2=0x7fda700e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7d204640 1 --2- 172.21.15.106:0/3532399095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700eb920 0x7fda700e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7d204640 1 -- 172.21.15.106:0/3532399095 >> v1:172.21.15.134:6789/0 conn(0x7fda700539c0 legacy=0x7fda70053dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.593+0000 7fda7d204640 1 -- 172.21.15.106:0/3532399095 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda70034400 con 0x7fda70033ab0 2024-04-25T11:19:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2a89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc0b50f0 0x7f43bc0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2a89640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f43bc0b5500 0x7f43bc0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2a89640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43bc06efb0 con 0x7f43bc06e840 2024-04-25T11:19:33.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2a89640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f43bc06fbc0 con 0x7f43bc0b5500 2024-04-25T11:19:33.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2a89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f43bc06f290 con 0x7f43bc0b50f0 2024-04-25T11:19:33.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f106f937640 1 -- start start 2024-04-25T11:19:33.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2288640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f43bc0b5500 0x7f43bc0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2288640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f43bc0b5500 0x7f43bc0aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45292/0 (socket says 172.21.15.106:45292) 2024-04-25T11:19:33.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2288640 1 -- 172.21.15.106:0/3545441632 learned_addr learned my addr 172.21.15.106:0/3545441632 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f24c6ffd640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24b0004810 0x7f24b0004c20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009ae0 tx=0x7f24b801cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c40cbe8b3480c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44a5d9b640 1 -- 172.21.15.106:0/1385778370 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a00a5b70 con 0x7f44a80ce140 2024-04-25T11:19:33.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44a5d9b640 1 -- 172.21.15.106:0/1385778370 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a00a5d10 con 0x7f44a80ce140 2024-04-25T11:19:33.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44a5d9b640 1 -- 172.21.15.106:0/1385778370 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a00ad610 con 0x7f44a80ce140 2024-04-25T11:19:33.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fda7d204640 1 --2- 172.21.15.106:0/3532399095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda70033ec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda740097c0 tx=0x7fda7401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6e9f6c3f920b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c1286640 1 --2- 172.21.15.106:0/3545441632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc0b50f0 0x7f43bc0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b80b2070 con 0x7f24b0004810 2024-04-25T11:19:33.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b80a5550 con 0x7f24b0004810 2024-04-25T11:19:33.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f24c67fc640 1 -- 172.21.15.106:0/1954887067 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b80096b0 con 0x7f24b0004810 2024-04-25T11:19:33.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fda7ca03640 1 -- 172.21.15.106:0/3532399095 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740a7070 con 0x7fda70033ab0 2024-04-25T11:19:33.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fda7ca03640 1 -- 172.21.15.106:0/3532399095 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda740b0070 con 0x7fda70033ab0 2024-04-25T11:19:33.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fda7ca03640 1 -- 172.21.15.106:0/3532399095 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740ab890 con 0x7fda70033ab0 2024-04-25T11:19:33.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fb35182d640 1 --2- 172.21.15.106:0/3488638985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 0x7fb34c0d05e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0725f0 tx=0x7fb34401c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=108aff0ee8b2b9f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fcf837fe640 1 --2- 172.21.15.106:0/528802820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415d4d0 0x7fcf8415d8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf8406e360 tx=0x7fcf7c01ea10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cada0c74376a5618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2288640 1 -- 172.21.15.106:0/3545441632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc0b50f0 msgr2=0x7f43bc0ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2288640 1 --2- 172.21.15.106:0/3545441632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc0b50f0 0x7f43bc0ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2288640 1 -- 172.21.15.106:0/3545441632 >> v1:172.21.15.134:6789/0 conn(0x7f43bc06e840 legacy=0x7f43bc0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f43c2288640 1 -- 172.21.15.106:0/3545441632 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc06f570 con 0x7f43bc0b5500 2024-04-25T11:19:33.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44ac881640 1 -- 172.21.15.106:0/1385778370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44a80ce140 msgr2=0x7f44a80ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44ac881640 1 --2- 172.21.15.106:0/1385778370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44a80ce140 0x7f44a80ce550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009bd0 tx=0x7f44a001cdb0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44ac881640 1 -- 172.21.15.106:0/1385778370 shutdown_connections 2024-04-25T11:19:33.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44ac881640 1 --2- 172.21.15.106:0/1385778370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44a80ce140 0x7f44a80ce550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44ac881640 1 --2- 172.21.15.106:0/1385778370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80700a0 0x7f44a815c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44ac881640 1 -- 172.21.15.106:0/1385778370 >> 172.21.15.106:0/1385778370 conn(0x7f44a806ca20 msgr2=0x7f44a806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f106f937640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1068070110 0x7f10680cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f106f937640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1068160750 0x7f10680cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f106f937640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1068161200 con 0x7f1068160340 2024-04-25T11:19:33.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f106f937640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f10680cf5f0 con 0x7f1068160750 2024-04-25T11:19:33.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f106f937640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1068160c40 con 0x7f1068070110 2024-04-25T11:19:33.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fda7fc69640 1 -- 172.21.15.106:0/3532399095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 msgr2=0x7fda70033ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7fda7fc69640 1 --2- 172.21.15.106:0/3532399095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda70033ec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda740097c0 tx=0x7fda7401c470 comp rx=0 tx=0).stop 2024-04-25T11:19:33.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.597+0000 7f44ac881640 1 -- 172.21.15.106:0/1385778370 shutdown_connections 2024-04-25T11:19:33.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 -- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24b0004810 msgr2=0x7f24b0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24b0004810 0x7f24b0004c20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009ae0 tx=0x7f24b801cad0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 -- 172.21.15.106:0/1954887067 shutdown_connections 2024-04-25T11:19:33.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 --2- 172.21.15.106:0/1954887067 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24b0004810 0x7f24b0004c20 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009ae0 tx=0x7f24b801cad0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 -- 172.21.15.106:0/1954887067 >> 172.21.15.106:0/1954887067 conn(0x7f24c80b3770 msgr2=0x7f24c80d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb33ffff640 1 -- 172.21.15.106:0/3488638985 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3440a56a0 con 0x7fb34c0d01d0 2024-04-25T11:19:33.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb33ffff640 1 -- 172.21.15.106:0/3488638985 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3440a5840 con 0x7fb34c0d01d0 2024-04-25T11:19:33.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb33ffff640 1 -- 172.21.15.106:0/3488638985 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3440a5c10 con 0x7fb34c0d01d0 2024-04-25T11:19:33.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf82ffd640 1 -- 172.21.15.106:0/528802820 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0ad070 con 0x7fcf8415d4d0 2024-04-25T11:19:33.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf82ffd640 1 -- 172.21.15.106:0/528802820 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c0a7a60 con 0x7fcf8415d4d0 2024-04-25T11:19:33.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf82ffd640 1 -- 172.21.15.106:0/528802820 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0a7e30 con 0x7fcf8415d4d0 2024-04-25T11:19:33.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fda7fc69640 1 -- 172.21.15.106:0/3532399095 shutdown_connections 2024-04-25T11:19:33.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fda7fc69640 1 --2- 172.21.15.106:0/3532399095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda70033ec0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fda7fc69640 1 --2- 172.21.15.106:0/3532399095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700eb920 0x7fda700e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fda7fc69640 1 -- 172.21.15.106:0/3532399095 >> 172.21.15.106:0/3532399095 conn(0x7fda70054d30 msgr2=0x7fda70055140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fda7fc69640 1 -- 172.21.15.106:0/3532399095 shutdown_connections 2024-04-25T11:19:33.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 -- 172.21.15.106:0/1954887067 shutdown_connections 2024-04-25T11:19:33.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f106ced2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1068070110 0x7f10680cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fda7fc69640 1 -- 172.21.15.106:0/3532399095 wait complete. 2024-04-25T11:19:33.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f106d6d3640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f1068160340 0x7f106806fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:56522/0 (socket says 172.21.15.106:56522) 2024-04-25T11:19:33.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f106d6d3640 1 -- 172.21.15.106:0/3766388874 learned_addr learned my addr 172.21.15.106:0/3766388874 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:33.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f106ced2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1068070110 0x7f10680cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40216/0 (socket says 172.21.15.106:40216) 2024-04-25T11:19:33.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4d1528640 1 --2- 172.21.15.106:0/2972572698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc06d600 0x7ff4cc0b10b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0097c0 tx=0x7ff4bc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4b252f81c269ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4cb7fe640 1 -- 172.21.15.106:0/2972572698 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0b1070 con 0x7ff4cc06d600 2024-04-25T11:19:33.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f106ced2640 1 -- 172.21.15.106:0/3766388874 >> v2:172.21.15.106:3300/0 conn(0x7f1068160750 msgr2=0x7f10680cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f106ced2640 1 --2- 172.21.15.106:0/3766388874 >> v2:172.21.15.106:3300/0 conn(0x7f1068160750 0x7f10680cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f106ced2640 1 -- 172.21.15.106:0/3766388874 >> v1:172.21.15.134:6789/0 conn(0x7f1068160340 legacy=0x7f106806fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f106ced2640 1 -- 172.21.15.106:0/3766388874 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1068160f20 con 0x7f1068070110 2024-04-25T11:19:33.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf894a8640 1 -- 172.21.15.106:0/528802820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415d4d0 msgr2=0x7fcf8415d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf894a8640 1 --2- 172.21.15.106:0/528802820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415d4d0 0x7fcf8415d8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf8406e360 tx=0x7fcf7c01ea10 comp rx=0 tx=0).stop 2024-04-25T11:19:33.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4cb7fe640 1 -- 172.21.15.106:0/2972572698 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4bc0ac5f0 con 0x7ff4cc06d600 2024-04-25T11:19:33.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4cb7fe640 1 -- 172.21.15.106:0/2972572698 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0ac9c0 con 0x7ff4cc06d600 2024-04-25T11:19:33.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/2972572698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc06d600 msgr2=0x7ff4cc0b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4d2f8b640 1 --2- 172.21.15.106:0/2972572698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc06d600 0x7ff4cc0b10b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0097c0 tx=0x7ff4bc01c4a0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f44ac881640 1 -- 172.21.15.106:0/1385778370 wait complete. 2024-04-25T11:19:33.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f44ac881640 1 Processor -- start 2024-04-25T11:19:33.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/2972572698 shutdown_connections 2024-04-25T11:19:33.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4d2f8b640 1 --2- 172.21.15.106:0/2972572698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc06d600 0x7ff4cc0b10b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/2972572698 >> 172.21.15.106:0/2972572698 conn(0x7ff4cc073c00 msgr2=0x7ff4cc0b7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/2972572698 shutdown_connections 2024-04-25T11:19:33.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb35202e640 1 -- 172.21.15.106:0/3488638985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 msgr2=0x7fb34c0d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb35202e640 1 --2- 172.21.15.106:0/3488638985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 0x7fb34c0d05e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0725f0 tx=0x7fb34401c8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/2972572698 wait complete. 2024-04-25T11:19:33.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f44ac881640 1 -- start start 2024-04-25T11:19:33.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 -- 172.21.15.106:0/1954887067 wait complete. 2024-04-25T11:19:33.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb35202e640 1 -- 172.21.15.106:0/3488638985 shutdown_connections 2024-04-25T11:19:33.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb35202e640 1 --2- 172.21.15.106:0/3488638985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 0x7fb34c0d05e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb35202e640 1 --2- 172.21.15.106:0/3488638985 >> v2:172.21.15.106:3300/0 conn(0x7fb34c06e8b0 0x7fb34c06ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb35202e640 1 -- 172.21.15.106:0/3488638985 >> 172.21.15.106:0/3488638985 conn(0x7fb34c0b3770 msgr2=0x7fb34c0d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf894a8640 1 -- 172.21.15.106:0/528802820 shutdown_connections 2024-04-25T11:19:33.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf894a8640 1 --2- 172.21.15.106:0/528802820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415d4d0 0x7fcf8415d8e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf894a8640 1 --2- 172.21.15.106:0/528802820 >> v2:172.21.15.106:3300/0 conn(0x7fcf8415f2d0 0x7fcf84152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fcf894a8640 1 -- 172.21.15.106:0/528802820 >> 172.21.15.106:0/528802820 conn(0x7fcf84073c10 msgr2=0x7fcf84074020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb35202e640 1 -- 172.21.15.106:0/3488638985 shutdown_connections 2024-04-25T11:19:33.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fda7fc69640 1 Processor -- start 2024-04-25T11:19:33.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 Processor -- start 2024-04-25T11:19:33.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f24ccefc640 1 -- start start 2024-04-25T11:19:33.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fda7fc69640 1 -- start start 2024-04-25T11:19:33.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f43c2288640 1 --2- 172.21.15.106:0/3545441632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b5500 0x7f43bc0aea70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43b8008870 tx=0x7f43b801e9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7009698133b73d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f43c0a85640 1 -- 172.21.15.106:0/3545441632 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80ad070 con 0x7f43bc0b5500 2024-04-25T11:19:33.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7f43c0a85640 1 -- 172.21.15.106:0/3545441632 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b80a79c0 con 0x7f43bc0b5500 2024-04-25T11:19:33.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44a80700a0 0x7f44a81fd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80ce140 0x7f44a8072680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44a815ce10 0x7f44a8165ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80b48d0 con 0x7f44a815ce10 2024-04-25T11:19:33.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80b4a50 con 0x7f44a80700a0 2024-04-25T11:19:33.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f44a8168f80 con 0x7f44a80ce140 2024-04-25T11:19:33.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fcf894a8640 1 -- 172.21.15.106:0/528802820 shutdown_connections 2024-04-25T11:19:33.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fcf894a8640 1 -- 172.21.15.106:0/528802820 wait complete. 2024-04-25T11:19:33.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f43c2a89640 1 -- 172.21.15.106:0/3545441632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b5500 msgr2=0x7f43bc0aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3545441632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b5500 0x7f43bc0aea70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43b8008870 tx=0x7f43b801e9e0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24ccefc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c8160410 0x7f24c8154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24ccefc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24c80af1f0 0x7f24c80af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7ff4d2f8b640 1 Processor -- start 2024-04-25T11:19:33.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24ccefc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c81563f0 0x7f24c8156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106ced2640 1 --2- 172.21.15.106:0/3766388874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1068070110 0x7f10680cfa80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10640097f0 tx=0x7f106401e6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9361c62922aca592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f105ffff640 1 -- 172.21.15.106:0/3766388874 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640ac650 con 0x7f1068070110 2024-04-25T11:19:33.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f105ffff640 1 -- 172.21.15.106:0/3766388874 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10640b1070 con 0x7f1068070110 2024-04-25T11:19:33.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f105ffff640 1 -- 172.21.15.106:0/3766388874 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640aca10 con 0x7f1068070110 2024-04-25T11:19:33.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7ff4d2f8b640 1 -- start start 2024-04-25T11:19:33.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c81563f0 0x7f24c8156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c8160410 0x7f24c8154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c8160410 0x7f24c8154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45310/0 (socket says 172.21.15.106:45310) 2024-04-25T11:19:33.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c77fe640 1 -- 172.21.15.106:0/628287842 learned_addr learned my addr 172.21.15.106:0/628287842 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24c80af1f0 0x7f24c80af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24ccefc640 1 -- 172.21.15.106:0/628287842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8073160 con 0x7f24c81563f0 2024-04-25T11:19:33.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24ccefc640 1 -- 172.21.15.106:0/628287842 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80732e0 con 0x7f24c8160410 2024-04-25T11:19:33.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24ccefc640 1 -- 172.21.15.106:0/628287842 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f24c8073460 con 0x7f24c80af1f0 2024-04-25T11:19:33.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106f937640 1 -- 172.21.15.106:0/3766388874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1068070110 msgr2=0x7f10680cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106f937640 1 --2- 172.21.15.106:0/3766388874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1068070110 0x7f10680cfa80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10640097f0 tx=0x7f106401e6f0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f43c2a89640 1 -- 172.21.15.106:0/3545441632 shutdown_connections 2024-04-25T11:19:33.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3545441632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b5500 0x7f43bc0aea70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3545441632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc0b50f0 0x7f43bc0ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f43c2a89640 1 -- 172.21.15.106:0/3545441632 >> 172.21.15.106:0/3545441632 conn(0x7f43bc0b3770 msgr2=0x7f43bc0d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106f937640 1 -- 172.21.15.106:0/3766388874 shutdown_connections 2024-04-25T11:19:33.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106f937640 1 --2- 172.21.15.106:0/3766388874 >> v2:172.21.15.106:3300/0 conn(0x7f1068160750 0x7f10680cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106f937640 1 --2- 172.21.15.106:0/3766388874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1068070110 0x7f10680cfa80 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10640097f0 tx=0x7f106401e6f0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106f937640 1 -- 172.21.15.106:0/3766388874 >> 172.21.15.106:0/3766388874 conn(0x7f106806cbf0 msgr2=0x7f10680d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106f937640 1 -- 172.21.15.106:0/3766388874 shutdown_connections 2024-04-25T11:19:33.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.601+0000 7fb35202e640 1 -- 172.21.15.106:0/3488638985 wait complete. 2024-04-25T11:19:33.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fb35202e640 1 Processor -- start 2024-04-25T11:19:33.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f106f937640 1 -- 172.21.15.106:0/3766388874 wait complete. 2024-04-25T11:19:33.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f43c2a89640 1 -- 172.21.15.106:0/3545441632 shutdown_connections 2024-04-25T11:19:33.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44a80700a0 0x7f44a81fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44a80700a0 0x7f44a81fd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45302/0 (socket says 172.21.15.106:45302) 2024-04-25T11:19:33.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 -- 172.21.15.106:0/4244998271 learned_addr learned my addr 172.21.15.106:0/4244998271 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 -- 172.21.15.106:0/4244998271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80ce140 msgr2=0x7f44a8072680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80ce140 0x7f44a8072680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 -- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44a815ce10 msgr2=0x7f44a8165ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44a815ce10 0x7f44a8165ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 -- 172.21.15.106:0/4244998271 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a8073290 con 0x7f44a80700a0 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44a6d9d640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44a80700a0 0x7f44a81fd970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f449c00fa10 tx=0x7f449c00fed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f448f7fe640 1 -- 172.21.15.106:0/4244998271 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c00daf0 con 0x7f44a80700a0 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f448f7fe640 1 -- 172.21.15.106:0/4244998271 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449c00dc90 con 0x7f44a80700a0 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f448f7fe640 1 -- 172.21.15.106:0/4244998271 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c01a6c0 con 0x7f44a80700a0 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a8073570 con 0x7f44a80700a0 2024-04-25T11:19:33.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a80b0ed0 con 0x7f44a80700a0 2024-04-25T11:19:33.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c7fff640 1 -- 172.21.15.106:0/628287842 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24c80af1f0 msgr2=0x7f24c80af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c7fff640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24c80af1f0 0x7f24c80af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c7fff640 1 -- 172.21.15.106:0/628287842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c8160410 msgr2=0x7f24c8154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c7fff640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c8160410 0x7f24c8154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c7fff640 1 -- 172.21.15.106:0/628287842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80afb40 con 0x7f24c81563f0 2024-04-25T11:19:33.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fcf894a8640 1 Processor -- start 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7fc69640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda700e7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7da05640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda700e7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7da05640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda700e7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46322/0 (socket says 172.21.15.106:46322) 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7da05640 1 -- 172.21.15.106:0/2644958800 learned_addr learned my addr 172.21.15.106:0/2644958800 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7fc69640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700539c0 0x7fda70048a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7fc69640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda700eb920 0x7fda700e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda700ebd30 con 0x7fda70033ab0 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda700e8180 con 0x7fda700eb920 2024-04-25T11:19:33.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fda700ebeb0 con 0x7fda700539c0 2024-04-25T11:19:33.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a8150c20 con 0x7f44a80700a0 2024-04-25T11:19:33.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7f24c7fff640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c81563f0 0x7f24c8156800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f24c8072620 tx=0x7f24bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f24affff640 1 -- 172.21.15.106:0/628287842 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc007440 con 0x7f24c81563f0 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d2f8b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4cc06d600 0x7ff4cc1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d2f8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0d29b0 0x7ff4cc151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4cc06d600 0x7ff4cc1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4cc06d600 0x7ff4cc1530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46338/0 (socket says 172.21.15.106:46338) 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 -- 172.21.15.106:0/1474928865 learned_addr learned my addr 172.21.15.106:0/1474928865 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d2f8b640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc15cbb0 0x7ff4cc14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/1474928865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc16dd80 con 0x7ff4cc06d600 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/1474928865 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc16dc00 con 0x7ff4cc0d29b0 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/1474928865 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc16df00 con 0x7ff4cc15cbb0 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4cbfff640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0d29b0 0x7ff4cc151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d1528640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc15cbb0 0x7ff4cc14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 -- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc15cbb0 msgr2=0x7ff4cc14d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc15cbb0 0x7ff4cc14d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 -- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0d29b0 msgr2=0x7ff4cc151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0d29b0 0x7ff4cc151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 -- 172.21.15.106:0/1474928865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc153600 con 0x7ff4cc06d600 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d0d27640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4cc06d600 0x7ff4cc1530c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000c370 tx=0x7ff4c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4c97fa640 1 -- 172.21.15.106:0/1474928865 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0017070 con 0x7ff4cc06d600 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4c97fa640 1 -- 172.21.15.106:0/1474928865 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c00052d0 con 0x7ff4cc06d600 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4c97fa640 1 -- 172.21.15.106:0/1474928865 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0003840 con 0x7ff4cc06d600 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/1474928865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc14cea0 con 0x7ff4cc06d600 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/1474928865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc159760 con 0x7ff4cc06d600 2024-04-25T11:19:33.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f43c2a89640 1 -- 172.21.15.106:0/3545441632 wait complete. 2024-04-25T11:19:33.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f43c2a89640 1 Processor -- start 2024-04-25T11:19:33.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f43c2a89640 1 -- start start 2024-04-25T11:19:33.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f448f7fe640 1 -- 172.21.15.106:0/4244998271 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f449c004020 con 0x7f44a80700a0 2024-04-25T11:19:33.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f448f7fe640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4494080440 0x7f4494082950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f448f7fe640 1 -- 172.21.15.106:0/4244998271 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f449c08fc60 con 0x7f44a80700a0 2024-04-25T11:19:33.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f44a659c640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4494080440 0x7f4494082950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f24affff640 1 -- 172.21.15.106:0/628287842 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24bc0075e0 con 0x7f24c81563f0 2024-04-25T11:19:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f24ccefc640 1 -- 172.21.15.106:0/628287842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c8160c60 con 0x7f24c81563f0 2024-04-25T11:19:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f24ccefc640 1 -- 172.21.15.106:0/628287842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c8155630 con 0x7f24c81563f0 2024-04-25T11:19:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f24affff640 1 -- 172.21.15.106:0/628287842 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc004a30 con 0x7f24c81563f0 2024-04-25T11:19:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f24affff640 1 -- 172.21.15.106:0/628287842 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f24bc004bd0 con 0x7f24c81563f0 2024-04-25T11:19:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fb35202e640 1 -- start start 2024-04-25T11:19:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35202e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34c06e8b0 0x7fb34c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34c06e8b0 0x7fb34c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34c06e8b0 0x7fb34c15a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46334/0 (socket says 172.21.15.106:46334) 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 -- 172.21.15.106:0/463458815 learned_addr learned my addr 172.21.15.106:0/463458815 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35202e640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34c072d50 0x7fb34c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35202e640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 0x7fb34c1643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35202e640 1 -- 172.21.15.106:0/463458815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c0d0d30 con 0x7fb34c06e8b0 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35202e640 1 -- 172.21.15.106:0/463458815 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c0b4d40 con 0x7fb34c0d01d0 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35082b640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34c072d50 0x7fb34c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35202e640 1 -- 172.21.15.106:0/463458815 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb34c1fe440 con 0x7fb34c072d50 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35182d640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 0x7fb34c1643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 -- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34c072d50 msgr2=0x7fb34c1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34c072d50 0x7fb34c1556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 -- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 msgr2=0x7fb34c1643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 0x7fb34c1643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 -- 172.21.15.106:0/463458815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c15b300 con 0x7fb34c06e8b0 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35102c640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34c06e8b0 0x7fb34c15a920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb34800c410 tx=0x7fb34800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb33dffb640 1 -- 172.21.15.106:0/463458815 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348017070 con 0x7fb34c06e8b0 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb33dffb640 1 -- 172.21.15.106:0/463458815 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb348003600 con 0x7fb34c06e8b0 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb33dffb640 1 -- 172.21.15.106:0/463458815 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480039d0 con 0x7fb34c06e8b0 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35202e640 1 -- 172.21.15.106:0/463458815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34c15b5e0 con 0x7fb34c06e8b0 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fb35202e640 1 -- 172.21.15.106:0/463458815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34c15bba0 con 0x7fb34c06e8b0 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fcf894a8640 1 -- start start 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf894a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf84159e30 0x7fcf840cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf894a8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415d4d0 0x7fcf841584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf894a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415f2d0 0x7fcf84158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415d4d0 0x7fcf841584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415d4d0 0x7fcf841584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40276/0 (socket says 172.21.15.106:40276) 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf84159e30 0x7fcf840cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf84159e30 0x7fcf840cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46352/0 (socket says 172.21.15.106:46352) 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf837fe640 1 -- 172.21.15.106:0/3481480090 learned_addr learned my addr 172.21.15.106:0/3481480090 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8415dfa0 con 0x7fcf84159e30 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8415de20 con 0x7fcf8415f2d0 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8415a950 con 0x7fcf8415d4d0 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf88ca7640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415f2d0 0x7fcf84158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf88ca7640 1 -- 172.21.15.106:0/3481480090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415d4d0 msgr2=0x7fcf841584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf88ca7640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415d4d0 0x7fcf841584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf88ca7640 1 -- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf84159e30 msgr2=0x7fcf840cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf88ca7640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf84159e30 0x7fcf840cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fcf88ca7640 1 -- 172.21.15.106:0/3481480090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf841fa8c0 con 0x7fcf8415f2d0 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fcf88ca7640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415f2d0 0x7fcf84158a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800ec70 tx=0x7fcf7800c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7d204640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700539c0 0x7fda70048a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.605+0000 7fda7e206640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda700eb920 0x7fda700e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7d204640 1 -- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda700eb920 msgr2=0x7fda700e9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7d204640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda700eb920 0x7fda700e9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7d204640 1 -- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 msgr2=0x7fda700e7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7d204640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda700e7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7d204640 1 -- 172.21.15.106:0/2644958800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda700e7ec0 con 0x7fda700539c0 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7da05640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda700e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7d204640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700539c0 0x7fda70048a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda740ac000 tx=0x7fda7401acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda6e7fc640 1 -- 172.21.15.106:0/2644958800 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740bf610 con 0x7fda700539c0 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7004ac00 con 0x7fda700539c0 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda6e7fc640 1 -- 172.21.15.106:0/2644958800 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda740bf7b0 con 0x7fda700539c0 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda6e7fc640 1 -- 172.21.15.106:0/2644958800 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740bfba0 con 0x7fda700539c0 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7004b1c0 con 0x7fda700539c0 2024-04-25T11:19:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f106f937640 1 Processor -- start 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.609+0000 7f106f937640 1 -- start start 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106f937640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1068160340 0x7f10680ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106f937640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1068160750 0x7f10680b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ced2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1068160750 0x7f10680b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ced2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1068160750 0x7f10680b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46366/0 (socket says 172.21.15.106:46366) 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106d6d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1068160340 0x7f10680ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ced2640 1 -- 172.21.15.106:0/1204466135 learned_addr learned my addr 172.21.15.106:0/1204466135 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106f937640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10680b14e0 0x7f10680af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f106806ec80 con 0x7f1068160750 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f106806eb00 con 0x7f1068160340 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f106806ee00 con 0x7f10680b14e0 2024-04-25T11:19:33.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ded4640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10680b14e0 0x7f10680af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fb35202e640 1 -- 172.21.15.106:0/463458815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb32c0052a0 con 0x7fb34c06e8b0 2024-04-25T11:19:33.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f24affff640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f24b406f400 0x7f24b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f24affff640 1 -- 172.21.15.106:0/628287842 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f24bc08ea70 con 0x7f24c81563f0 2024-04-25T11:19:33.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f24ccefc640 1 -- 172.21.15.106:0/628287842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a80052a0 con 0x7f24c81563f0 2024-04-25T11:19:33.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fda6e7fc640 1 -- 172.21.15.106:0/2644958800 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda740bd020 con 0x7fda700539c0 2024-04-25T11:19:33.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f24c77fe640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f24b406f400 0x7f24b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/1474928865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc050750 con 0x7ff4cc06d600 2024-04-25T11:19:33.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fda6e7fc640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda5c06f400 0x7fda5c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2a89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc06e840 0x7f43bc200790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2a89640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b50f0 0x7f43bc0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2a89640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43bc0b5500 0x7f43bc0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2a89640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc168ca0 con 0x7f43bc0b5500 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2a89640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc168b20 con 0x7f43bc0b50f0 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2a89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f43bc168e20 con 0x7f43bc06e840 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c1a87640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc06e840 0x7f43bc200790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c1a87640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc06e840 0x7f43bc200790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40284/0 (socket says 172.21.15.106:40284) 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c1a87640 1 -- 172.21.15.106:0/3492241882 learned_addr learned my addr 172.21.15.106:0/3492241882 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c1286640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b50f0 0x7f43bc0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2288640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43bc0b5500 0x7f43bc0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2288640 1 -- 172.21.15.106:0/3492241882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc06e840 msgr2=0x7f43bc200790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2288640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc06e840 0x7f43bc200790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2288640 1 -- 172.21.15.106:0/3492241882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b50f0 msgr2=0x7f43bc0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2288640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b50f0 0x7f43bc0cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2288640 1 -- 172.21.15.106:0/3492241882 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc1fedc0 con 0x7f43bc0b5500 2024-04-25T11:19:33.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fda6e7fc640 1 -- 172.21.15.106:0/2644958800 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda74129840 con 0x7fda700539c0 2024-04-25T11:19:33.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f44a659c640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4494080440 0x7f4494082950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4498007fd0 tx=0x7f4498007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda700055b0 con 0x7fda700539c0 2024-04-25T11:19:33.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7ff4c97fa640 1 -- 172.21.15.106:0/1474928865 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4c001a070 con 0x7ff4cc06d600 2024-04-25T11:19:33.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fda7da05640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda5c06f400 0x7fda5c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f43c2288640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43bc0b5500 0x7f43bc0ce0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43b80af000 tx=0x7f43b801d220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f24c77fe640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f24b406f400 0x7f24b4071910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f24c815d990 tx=0x7f24c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7ff4c97fa640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff4b806f450 0x7ff4b8071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7ff4c97fa640 1 -- 172.21.15.106:0/1474928865 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff4c008e680 con 0x7ff4cc06d600 2024-04-25T11:19:33.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fda7da05640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda5c06f400 0x7fda5c071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda68007e40 tx=0x7fda68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7ff4cbfff640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff4b806f450 0x7ff4b8071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ded4640 1 -- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1068160340 msgr2=0x7f10680ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ded4640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1068160340 0x7f10680ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ded4640 1 -- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1068160750 msgr2=0x7f10680b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ded4640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1068160750 0x7f10680b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ded4640 1 -- 172.21.15.106:0/1204466135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10680cbea0 con 0x7f10680b14e0 2024-04-25T11:19:33.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7f106ded4640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10680b14e0 0x7f10680af9b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f106000bdf0 tx=0x7f106000bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.613+0000 7fb33dffb640 1 -- 172.21.15.106:0/463458815 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb34801a070 con 0x7fb34c06e8b0 2024-04-25T11:19:33.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.617+0000 7ff4cbfff640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff4b806f450 0x7ff4b8071960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc069600 tx=0x7ff4c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.617+0000 7fb33dffb640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb33006f480 0x7fb330071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.617+0000 7fb33dffb640 1 -- 172.21.15.106:0/463458815 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb34808e8e0 con 0x7fb34c06e8b0 2024-04-25T11:19:33.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.617+0000 7fb35082b640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb33006f480 0x7fb330071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.617+0000 7fb35082b640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb33006f480 0x7fb330071990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb34c165000 tx=0x7fb340008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.621+0000 7fb33dffb640 1 -- 172.21.15.106:0/463458815 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb348021020 con 0x7fb34c06e8b0 2024-04-25T11:19:33.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7ff4c97fa640 1 -- 172.21.15.106:0/1474928865 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff4c0005480 con 0x7ff4cc06d600 2024-04-25T11:19:33.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7f43b27fc640 1 -- 172.21.15.106:0/3492241882 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80ad040 con 0x7f43bc0b5500 2024-04-25T11:19:33.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc15b040 con 0x7f43bc0b5500 2024-04-25T11:19:33.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc158c10 con 0x7f43bc0b5500 2024-04-25T11:19:33.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7fcf80ff9640 1 -- 172.21.15.106:0/3481480090 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78018030 con 0x7fcf8415f2d0 2024-04-25T11:19:33.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7fcf80ff9640 1 -- 172.21.15.106:0/3481480090 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7800cd50 con 0x7fcf8415f2d0 2024-04-25T11:19:33.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7fcf80ff9640 1 -- 172.21.15.106:0/3481480090 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78004260 con 0x7fcf8415f2d0 2024-04-25T11:19:33.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf841fadc0 con 0x7fcf8415f2d0 2024-04-25T11:19:33.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf841fb380 con 0x7fcf8415f2d0 2024-04-25T11:19:33.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7f448f7fe640 1 -- 172.21.15.106:0/4244998271 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f449c05cbd0 con 0x7f44a80700a0 2024-04-25T11:19:33.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43bc1fef90 con 0x7f43bc0b5500 2024-04-25T11:19:33.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.625+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf84050750 con 0x7fcf8415f2d0 2024-04-25T11:19:33.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.629+0000 7fcf80ff9640 1 -- 172.21.15.106:0/3481480090 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcf7801b070 con 0x7fcf8415f2d0 2024-04-25T11:19:33.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.629+0000 7fcf80ff9640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcf7006f400 0x7fcf70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.629+0000 7fcf80ff9640 1 -- 172.21.15.106:0/3481480090 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf78091c80 con 0x7fcf8415f2d0 2024-04-25T11:19:33.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.629+0000 7fcf83fff640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcf7006f400 0x7fcf70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.629+0000 7fcf83fff640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcf7006f400 0x7fcf70071910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf84071ff0 tx=0x7fcf74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.633+0000 7f43b27fc640 1 -- 172.21.15.106:0/3492241882 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b80b1de0 con 0x7f43bc0b5500 2024-04-25T11:19:33.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.633+0000 7f43b27fc640 1 -- 172.21.15.106:0/3492241882 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8008be0 con 0x7f43bc0b5500 2024-04-25T11:19:33.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.633+0000 7f24affff640 1 -- 172.21.15.106:0/628287842 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24c80afb40 con 0x7f24c81563f0 2024-04-25T11:19:33.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.637+0000 7f43b27fc640 1 -- 172.21.15.106:0/3492241882 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43b80c2060 con 0x7f43bc0b5500 2024-04-25T11:19:33.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.637+0000 7f43b27fc640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43a006f430 0x7f43a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.637+0000 7f43b27fc640 1 -- 172.21.15.106:0/3492241882 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f43b8128ee0 con 0x7f43bc0b5500 2024-04-25T11:19:33.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.637+0000 7f43c1a87640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43a006f430 0x7f43a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.637+0000 7f43c1a87640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43a006f430 0x7f43a0071940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0016f0 tx=0x7f43ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.641+0000 7fcf80ff9640 1 -- 172.21.15.106:0/3481480090 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf7805e620 con 0x7fcf8415f2d0 2024-04-25T11:19:33.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.645+0000 7f43b27fc640 1 -- 172.21.15.106:0/3492241882 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f43b8018660 con 0x7f43bc0b5500 2024-04-25T11:19:33.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.649+0000 7f105dffb640 1 -- 172.21.15.106:0/1204466135 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060019070 con 0x7f10680b14e0 2024-04-25T11:19:33.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.649+0000 7f105dffb640 1 -- 172.21.15.106:0/1204466135 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1060014440 con 0x7f10680b14e0 2024-04-25T11:19:33.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.649+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10680cc180 con 0x7f10680b14e0 2024-04-25T11:19:33.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.649+0000 7f105dffb640 1 -- 172.21.15.106:0/1204466135 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060014830 con 0x7f10680b14e0 2024-04-25T11:19:33.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.649+0000 7fda6e7fc640 1 -- 172.21.15.106:0/2644958800 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fda70034400 con 0x7fda700539c0 2024-04-25T11:19:33.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.649+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f10680cc740 con 0x7f10680b14e0 2024-04-25T11:19:33.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.649+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10680cbc10 con 0x7f10680b14e0 2024-04-25T11:19:33.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.653+0000 7f105dffb640 1 -- 172.21.15.106:0/1204466135 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f106001c070 con 0x7f10680b14e0 2024-04-25T11:19:33.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.653+0000 7f105dffb640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f104c06f400 0x7f104c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.653+0000 7f105dffb640 1 -- 172.21.15.106:0/1204466135 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f106008fa70 con 0x7f10680b14e0 2024-04-25T11:19:33.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.653+0000 7f106d6d3640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f104c06f400 0x7f104c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.653+0000 7f106d6d3640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f104c06f400 0x7f104c071910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1058000f70 tx=0x7f1058009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.665+0000 7f105dffb640 1 -- 172.21.15.106:0/1204466135 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f106005c2f0 con 0x7f10680b14e0 2024-04-25T11:19:33.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.937+0000 7fb35202e640 1 -- 172.21.15.106:0/463458815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb32c005860 con 0x7fb34c06e8b0 2024-04-25T11:19:33.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.937+0000 7fb33dffb640 1 -- 172.21.15.106:0/463458815 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb348001070 con 0x7fb34c06e8b0 2024-04-25T11:19:33.938 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:33.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 -- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb33006f480 msgr2=0x7fb330071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb33006f480 0x7fb330071990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb34c165000 tx=0x7fb340008040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 -- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34c06e8b0 msgr2=0x7fb34c15a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34c06e8b0 0x7fb34c15a920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb34800c410 tx=0x7fb34800c8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 -- 172.21.15.106:0/463458815 shutdown_connections 2024-04-25T11:19:33.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb34c0d01d0 0x7fb34c1643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34c072d50 0x7fb34c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb33006f480 0x7fb330071990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 --2- 172.21.15.106:0/463458815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34c06e8b0 0x7fb34c15a920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 -- 172.21.15.106:0/463458815 >> 172.21.15.106:0/463458815 conn(0x7fb34c0b3770 msgr2=0x7fb34c156250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 -- 172.21.15.106:0/463458815 shutdown_connections 2024-04-25T11:19:33.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.945+0000 7fb3377fe640 1 -- 172.21.15.106:0/463458815 wait complete. 2024-04-25T11:19:33.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.949+0000 7ff4d2f8b640 1 -- 172.21.15.106:0/1474928865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff4cc068710 con 0x7ff4cc06d600 2024-04-25T11:19:33.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.949+0000 7ff4c97fa640 1 -- 172.21.15.106:0/1474928865 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4c005b550 con 0x7ff4cc06d600 2024-04-25T11:19:33.947 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:33.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 -- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff4b806f450 msgr2=0x7ff4b8071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff4b806f450 0x7ff4b8071960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc069600 tx=0x7ff4c4009040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 -- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4cc06d600 msgr2=0x7ff4cc1530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4cc06d600 0x7ff4cc1530c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000c370 tx=0x7ff4c000c830 comp rx=0 tx=0).stop 2024-04-25T11:19:33.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f44a8050750 con 0x7f44a80700a0 2024-04-25T11:19:33.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 -- 172.21.15.106:0/1474928865 shutdown_connections 2024-04-25T11:19:33.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc15cbb0 0x7ff4cc14d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0d29b0 0x7ff4cc151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff4b806f450 0x7ff4b8071960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 --2- 172.21.15.106:0/1474928865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4cc06d600 0x7ff4cc1530c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 -- 172.21.15.106:0/1474928865 >> 172.21.15.106:0/1474928865 conn(0x7ff4cc073c00 msgr2=0x7ff4cc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 -- 172.21.15.106:0/1474928865 shutdown_connections 2024-04-25T11:19:33.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7ff4b2ffd640 1 -- 172.21.15.106:0/1474928865 wait complete. 2024-04-25T11:19:33.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7f448f7fe640 1 -- 172.21.15.106:0/4244998271 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f449c05c4d0 con 0x7f44a80700a0 2024-04-25T11:19:33.953 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T11:19:33.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7f24ccefc640 1 -- 172.21.15.106:0/628287842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f24a8005590 con 0x7f24c81563f0 2024-04-25T11:19:33.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.953+0000 7f24affff640 1 -- 172.21.15.106:0/628287842 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24bc05b300 con 0x7f24c81563f0 2024-04-25T11:19:33.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4494080440 msgr2=0x7f4494082950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4494080440 0x7f4494082950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4498007fd0 tx=0x7f4498007960 comp rx=0 tx=0).stop 2024-04-25T11:19:33.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44a80700a0 msgr2=0x7f44a81fd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44a80700a0 0x7f44a81fd970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f449c00fa10 tx=0x7f449c00fed0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 shutdown_connections 2024-04-25T11:19:33.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4494080440 0x7f4494082950 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4498007fd0 tx=0x7f4498007960 comp rx=0 tx=0).stop 2024-04-25T11:19:33.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44a815ce10 0x7f44a8165ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44a80ce140 0x7f44a8072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 --2- 172.21.15.106:0/4244998271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44a80700a0 0x7f44a81fd970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 >> 172.21.15.106:0/4244998271 conn(0x7f44a806ca20 msgr2=0x7f44a80b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.956 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 -- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f24b406f400 msgr2=0x7f24b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f24b406f400 0x7f24b4071910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f24c815d990 tx=0x7f24c0009040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 -- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c81563f0 msgr2=0x7f24c8156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c81563f0 0x7f24c8156800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f24c8072620 tx=0x7f24bc00c8e0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 -- 172.21.15.106:0/628287842 shutdown_connections 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f24b406f400 0x7f24b4071910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c81563f0 0x7f24c8156800 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24c80af1f0 0x7f24c80af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 --2- 172.21.15.106:0/628287842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c8160410 0x7f24c8154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 -- 172.21.15.106:0/628287842 >> 172.21.15.106:0/628287842 conn(0x7f24c80b3770 msgr2=0x7f24c8163560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 shutdown_connections 2024-04-25T11:19:33.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f44ac881640 1 -- 172.21.15.106:0/4244998271 wait complete. 2024-04-25T11:19:33.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 -- 172.21.15.106:0/628287842 shutdown_connections 2024-04-25T11:19:33.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.957+0000 7f24adffb640 1 -- 172.21.15.106:0/628287842 wait complete. 2024-04-25T11:19:33.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.961+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f43bc050750 con 0x7f43bc0b5500 2024-04-25T11:19:33.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.965+0000 7f43b27fc640 1 -- 172.21.15.106:0/3492241882 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f43b80f58a0 con 0x7f43bc0b5500 2024-04-25T11:19:33.965 INFO:teuthology.orchestra.run.smithi106.stdout:55834574853 2024-04-25T11:19:33.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.965+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fda70053250 con 0x7fda700539c0 2024-04-25T11:19:33.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.965+0000 7fda6e7fc640 1 -- 172.21.15.106:0/2644958800 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fda74016090 con 0x7fda700539c0 2024-04-25T11:19:33.965 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T11:19:33.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fcf8406d4a0 con 0x7fcf8415f2d0 2024-04-25T11:19:33.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43a006f430 msgr2=0x7f43a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43a006f430 0x7f43a0071940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0016f0 tx=0x7f43ac009040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43bc0b5500 msgr2=0x7f43bc0ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43bc0b5500 0x7f43bc0ce0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43b80af000 tx=0x7f43b801d220 comp rx=0 tx=0).stop 2024-04-25T11:19:33.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fcf80ff9640 1 -- 172.21.15.106:0/3481480090 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf78062920 con 0x7fcf8415f2d0 2024-04-25T11:19:33.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 shutdown_connections 2024-04-25T11:19:33.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43a006f430 0x7f43a0071940 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.968 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T11:19:33.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43bc0b5500 0x7f43bc0ce0a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc0b50f0 0x7f43bc0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 --2- 172.21.15.106:0/3492241882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43bc06e840 0x7f43bc200790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 >> 172.21.15.106:0/3492241882 conn(0x7f43bc0b3770 msgr2=0x7f43bc15c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 shutdown_connections 2024-04-25T11:19:33.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7f43c2a89640 1 -- 172.21.15.106:0/3492241882 wait complete. 2024-04-25T11:19:33.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda5c06f400 msgr2=0x7fda5c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda5c06f400 0x7fda5c071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda68007e40 tx=0x7fda68008040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700539c0 msgr2=0x7fda70048a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700539c0 0x7fda70048a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda740ac000 tx=0x7fda7401acb0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 shutdown_connections 2024-04-25T11:19:33.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda700eb920 0x7fda700e9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda700539c0 0x7fda70048a60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda5c06f400 0x7fda5c071910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 --2- 172.21.15.106:0/2644958800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda70033ab0 0x7fda700e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 >> 172.21.15.106:0/2644958800 conn(0x7fda70054d30 msgr2=0x7fda70053dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 shutdown_connections 2024-04-25T11:19:33.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.969+0000 7fda7fc69640 1 -- 172.21.15.106:0/2644958800 wait complete. 2024-04-25T11:19:33.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcf7006f400 msgr2=0x7fcf70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcf7006f400 0x7fcf70071910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf84071ff0 tx=0x7fcf74008040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415f2d0 msgr2=0x7fcf84158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415f2d0 0x7fcf84158a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800ec70 tx=0x7fcf7800c430 comp rx=0 tx=0).stop 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 shutdown_connections 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8415f2d0 0x7fcf84158a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf8415d4d0 0x7fcf841584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcf7006f400 0x7fcf70071910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 --2- 172.21.15.106:0/3481480090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf84159e30 0x7fcf840cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 >> 172.21.15.106:0/3481480090 conn(0x7fcf84073c10 msgr2=0x7fcf8406ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 shutdown_connections 2024-04-25T11:19:33.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.973+0000 7fcf894a8640 1 -- 172.21.15.106:0/3481480090 wait complete. 2024-04-25T11:19:33.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-04-25T11:19:33.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.985+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1068050750 con 0x7f10680b14e0 2024-04-25T11:19:33.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.985+0000 7f105dffb640 1 -- 172.21.15.106:0/1204466135 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f10600605f0 con 0x7f10680b14e0 2024-04-25T11:19:33.984 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T11:19:33.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f104c06f400 msgr2=0x7f104c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f104c06f400 0x7f104c071910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1058000f70 tx=0x7f1058009040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10680b14e0 msgr2=0x7f10680af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10680b14e0 0x7f10680af9b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f106000bdf0 tx=0x7f106000bee0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 shutdown_connections 2024-04-25T11:19:33.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10680b14e0 0x7f10680af9b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f104c06f400 0x7f104c071910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1068160750 0x7f10680b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 --2- 172.21.15.106:0/1204466135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1068160340 0x7f10680ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 >> 172.21.15.106:0/1204466135 conn(0x7f106806cbf0 msgr2=0x7f10680b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 shutdown_connections 2024-04-25T11:19:33.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:33.989+0000 7f106f937640 1 -- 172.21.15.106:0/1204466135 wait complete. 2024-04-25T11:19:33.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2024-04-25T11:19:33.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-25T11:19:33.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-25T11:19:34.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-25T11:19:34.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-04-25T11:19:34.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-04-25T11:19:34.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-25T11:19:34.978 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T11:19:34.990 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:19:34.995 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T11:19:34.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:19:35.001 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:19:35.005 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T11:19:35.007 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T11:19:35.012 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:19:35.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.205+0000 7f0eac309640 1 Processor -- start 2024-04-25T11:19:35.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.205+0000 7f0eac309640 1 -- start start 2024-04-25T11:19:35.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.205+0000 7f0eac309640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.205+0000 7f0eac309640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea40b2c60 con 0x7f0ea40b26f0 2024-04-25T11:19:35.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.205+0000 7f0eac309640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea40c6bb0 con 0x7f0ea40cbbd0 2024-04-25T11:19:35.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.205+0000 7f0eac309640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea40d0110 con 0x7f0ea40cb7c0 2024-04-25T11:19:35.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0ea98a4640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0ea40b26f0 0x7f0ea40d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57068/0 (socket says 172.21.15.106:57068) 2024-04-25T11:19:35.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0ea98a4640 1 -- 172.21.15.106:0/3711129922 learned_addr learned my addr 172.21.15.106:0/3711129922 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:35.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0eaa0a5640 1 --2- 172.21.15.106:0/3711129922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0ea90a3640 1 -- 172.21.15.106:0/3711129922 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 70948132 0 0) 0x7f0ea40c6bb0 con 0x7f0ea40cbbd0 2024-04-25T11:19:35.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0ea90a3640 1 -- 172.21.15.106:0/3711129922 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e90003530 con 0x7f0ea40cbbd0 2024-04-25T11:19:35.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0ea90a3640 1 -- 172.21.15.106:0/3711129922 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345580104 0 0) 0x7f0ea40b2c60 con 0x7f0ea40b26f0 2024-04-25T11:19:35.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0ea90a3640 1 -- 172.21.15.106:0/3711129922 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea40c6bb0 con 0x7f0ea40b26f0 2024-04-25T11:19:35.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0eaa0a5640 1 -- 172.21.15.106:0/3711129922 >> v1:172.21.15.134:6789/0 conn(0x7f0ea40cbbd0 legacy=0x7f0ea415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0eaa0a5640 1 -- 172.21.15.106:0/3711129922 >> v1:172.21.15.106:6789/0 conn(0x7f0ea40b26f0 legacy=0x7f0ea40d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.209+0000 7f0eaa0a5640 1 -- 172.21.15.106:0/3711129922 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea40cfac0 con 0x7f0ea40cb7c0 2024-04-25T11:19:35.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.213+0000 7f0eaa0a5640 1 --2- 172.21.15.106:0/3711129922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b21b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00097f0 tx=0x7f0ea001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4621c2d30c759887 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0ea90a3640 1 -- 172.21.15.106:0/3711129922 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00b0070 con 0x7f0ea40cb7c0 2024-04-25T11:19:35.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0ea90a3640 1 -- 172.21.15.106:0/3711129922 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea00ab600 con 0x7f0ea40cb7c0 2024-04-25T11:19:35.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0ea90a3640 1 -- 172.21.15.106:0/3711129922 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00ab9d0 con 0x7f0ea40cb7c0 2024-04-25T11:19:35.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0eac309640 1 -- 172.21.15.106:0/3711129922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 msgr2=0x7f0ea40b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0eac309640 1 --2- 172.21.15.106:0/3711129922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b21b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00097f0 tx=0x7f0ea001c450 comp rx=0 tx=0).stop 2024-04-25T11:19:35.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0eac309640 1 -- 172.21.15.106:0/3711129922 shutdown_connections 2024-04-25T11:19:35.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0eac309640 1 --2- 172.21.15.106:0/3711129922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b21b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0eac309640 1 -- 172.21.15.106:0/3711129922 >> 172.21.15.106:0/3711129922 conn(0x7f0ea40cdc80 msgr2=0x7f0ea40ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0eac309640 1 -- 172.21.15.106:0/3711129922 shutdown_connections 2024-04-25T11:19:35.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.221+0000 7f0eac309640 1 -- 172.21.15.106:0/3711129922 wait complete. 2024-04-25T11:19:35.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.225+0000 7f0eac309640 1 Processor -- start 2024-04-25T11:19:35.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.225+0000 7f0eac309640 1 -- start start 2024-04-25T11:19:35.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.225+0000 7f0eac309640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.225+0000 7f0eac309640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea40cbbd0 0x7f0ea409e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.225+0000 7f0eac309640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ea409ee40 0x7f0ea40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eac309640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea406e260 con 0x7f0ea409ee40 2024-04-25T11:19:35.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ea409ee40 0x7f0ea40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ea409ee40 0x7f0ea40b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46398/0 (socket says 172.21.15.106:46398) 2024-04-25T11:19:35.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 -- 172.21.15.106:0/1952552021 learned_addr learned my addr 172.21.15.106:0/1952552021 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eac309640 1 -- 172.21.15.106:0/1952552021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea406e0e0 con 0x7f0ea40cbbd0 2024-04-25T11:19:35.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0ea98a4640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea40cbbd0 0x7f0ea409e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eac309640 1 -- 172.21.15.106:0/1952552021 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea416a110 con 0x7f0ea40cb7c0 2024-04-25T11:19:35.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa0a5640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 -- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 msgr2=0x7f0ea40b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 -- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea40cbbd0 msgr2=0x7f0ea409e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea40cbbd0 0x7f0ea409e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 -- 172.21.15.106:0/1952552021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea4206970 con 0x7f0ea409ee40 2024-04-25T11:19:35.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa0a5640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:35.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eaa8a6640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ea409ee40 0x7f0ea40b4f30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c760 tx=0x7f0e9c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0e9affd640 1 -- 172.21.15.106:0/1952552021 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c003b80 con 0x7f0ea409ee40 2024-04-25T11:19:35.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0e9affd640 1 -- 172.21.15.106:0/1952552021 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9c003d20 con 0x7f0ea409ee40 2024-04-25T11:19:35.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0e9affd640 1 -- 172.21.15.106:0/1952552021 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00aa90 con 0x7f0ea409ee40 2024-04-25T11:19:35.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eac309640 1 -- 172.21.15.106:0/1952552021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea4205d60 con 0x7f0ea409ee40 2024-04-25T11:19:35.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.229+0000 7f0eac309640 1 -- 172.21.15.106:0/1952552021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea4206320 con 0x7f0ea409ee40 2024-04-25T11:19:35.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.233+0000 7f0e9affd640 1 -- 172.21.15.106:0/1952552021 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0e9c00ac30 con 0x7f0ea409ee40 2024-04-25T11:19:35.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.233+0000 7f0e9affd640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0e8806f4d0 0x7f0e880719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.233+0000 7f0e9affd640 1 -- 172.21.15.106:0/1952552021 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0e9c08ef90 con 0x7f0ea409ee40 2024-04-25T11:19:35.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.233+0000 7f0eaa0a5640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0e8806f4d0 0x7f0e880719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.233+0000 7f0eaa0a5640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0e8806f4d0 0x7f0e880719e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00ad000 tx=0x7f0ea00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.237+0000 7f0eac309640 1 -- 172.21.15.106:0/1952552021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c0052a0 con 0x7f0ea409ee40 2024-04-25T11:19:35.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.245+0000 7f0e9affd640 1 -- 172.21.15.106:0/1952552021 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e9c05bf30 con 0x7f0ea409ee40 2024-04-25T11:19:35.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7fd4cd0c3640 1 Processor -- start 2024-04-25T11:19:35.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7fd4cd0c3640 1 -- start start 2024-04-25T11:19:35.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7f227fafb640 1 Processor -- start 2024-04-25T11:19:35.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7f227fafb640 1 -- start start 2024-04-25T11:19:35.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7f4989c29640 1 Processor -- start 2024-04-25T11:19:35.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7fd4cd0c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c0057da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7fd4cd0c3640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c00586f0 con 0x7fd4c00582e0 2024-04-25T11:19:35.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7fd4cd0c3640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c0007910 con 0x7fd4c003e430 2024-04-25T11:19:35.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7fd4cd0c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c0007bf0 con 0x7fd4c0057620 2024-04-25T11:19:35.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7f4989c29640 1 -- start start 2024-04-25T11:19:35.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7fd4c77fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd4c00582e0 0x7fd4c00063b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57078/0 (socket says 172.21.15.106:57078) 2024-04-25T11:19:35.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.249+0000 7fd4c77fe640 1 -- 172.21.15.106:0/1280872664 learned_addr learned my addr 172.21.15.106:0/1280872664 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:35.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7fd4c7fff640 1 --2- 172.21.15.106:0/1280872664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c0057da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7fd4c6ffd640 1 -- 172.21.15.106:0/1280872664 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 494227578 0 0) 0x7fd4b4001120 con 0x7fd4c00582e0 2024-04-25T11:19:35.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7fd4c6ffd640 1 -- 172.21.15.106:0/1280872664 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4b8003530 con 0x7fd4c00582e0 2024-04-25T11:19:35.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5d0143640 1 Processor -- start 2024-04-25T11:19:35.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7fd4c6ffd640 1 -- 172.21.15.106:0/1280872664 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1709240026 0 0) 0x7fd4b8003530 con 0x7fd4c00582e0 2024-04-25T11:19:35.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7fd4c7fff640 1 -- 172.21.15.106:0/1280872664 >> v1:172.21.15.134:6789/0 conn(0x7fd4c003e430 legacy=0x7fd4c0006ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5d0143640 1 -- start start 2024-04-25T11:19:35.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227fafb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 0x7f227806fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227fafb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2278160750 0x7f227806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227fafb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f22780b2de0 0x7f22780b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227fafb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f22780cf5f0 con 0x7f2278160750 2024-04-25T11:19:35.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227fafb640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2278161200 con 0x7f22780b2de0 2024-04-25T11:19:35.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227fafb640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2278160c40 con 0x7f2278160340 2024-04-25T11:19:35.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7fd4c7fff640 1 -- 172.21.15.106:0/1280872664 >> v1:172.21.15.106:6789/0 conn(0x7fd4c00582e0 legacy=0x7fd4c00063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7fd4c7fff640 1 -- 172.21.15.106:0/1280872664 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c003cc20 con 0x7fd4c0057620 2024-04-25T11:19:35.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227d096640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2278160750 0x7f227806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227d096640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2278160750 0x7f227806ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46412/0 (socket says 172.21.15.106:46412) 2024-04-25T11:19:35.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227d096640 1 -- 172.21.15.106:0/637263274 learned_addr learned my addr 172.21.15.106:0/637263274 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4989c29640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f49840b4830 0x7f49840b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4989c29640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49840b1f90 0x7f49840b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4989c29640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49840b0140 con 0x7f49841605e0 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4989c29640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f49840aff10 con 0x7f49840b1f90 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4989c29640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f49840b5010 con 0x7f49840b4830 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4982ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f49841605e0 0x7f49841609f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57082/0 (socket says 172.21.15.106:57082) 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4982ffd640 1 -- 172.21.15.106:0/1546334456 learned_addr learned my addr 172.21.15.106:0/1546334456 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4983fff640 1 --2- 172.21.15.106:0/1546334456 >> v2:172.21.15.106:3300/0 conn(0x7f49840b1f90 0x7f49840b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4983fff640 1 -- 172.21.15.106:0/1546334456 >> v2:172.21.15.134:3300/0 conn(0x7f49840b4830 msgr2=0x7f49840b0f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4983fff640 1 --2- 172.21.15.106:0/1546334456 >> v2:172.21.15.134:3300/0 conn(0x7f49840b4830 0x7f49840b0f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4983fff640 1 -- 172.21.15.106:0/1546334456 >> v1:172.21.15.106:6789/0 conn(0x7f49841605e0 legacy=0x7f49841609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f4983fff640 1 -- 172.21.15.106:0/1546334456 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49840b0420 con 0x7f49840b1f90 2024-04-25T11:19:35.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5d0143640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff5c80b6030 0x7ff5c80b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5d0143640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c80b6540 con 0x7ff5c815ce30 2024-04-25T11:19:35.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5d0143640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c80b45e0 con 0x7ff5c8072ac0 2024-04-25T11:19:35.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5d0143640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c81532a0 con 0x7ff5c80b6030 2024-04-25T11:19:35.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5ce6e0640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff5c8072ac0 0x7ff5c8072ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:56556/0 (socket says 172.21.15.106:56556) 2024-04-25T11:19:35.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5ce6e0640 1 -- 172.21.15.106:0/989921716 learned_addr learned my addr 172.21.15.106:0/989921716 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:35.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f90f55c0640 1 Processor -- start 2024-04-25T11:19:35.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f062031a640 1 Processor -- start 2024-04-25T11:19:35.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f90f55c0640 1 -- start start 2024-04-25T11:19:35.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7ff5cdedf640 1 --2- 172.21.15.106:0/989921716 >> v2:172.21.15.134:3300/0 conn(0x7ff5c80b6030 0x7ff5c80b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4c7fff640 1 --2- 172.21.15.106:0/1280872664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c0057da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097f0 tx=0x7fd4bc01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573ebdccd3442459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4c6ffd640 1 -- 172.21.15.106:0/1280872664 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0b0070 con 0x7fd4c0057620 2024-04-25T11:19:35.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4c6ffd640 1 -- 172.21.15.106:0/1280872664 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc0ab400 con 0x7fd4c0057620 2024-04-25T11:19:35.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4c6ffd640 1 -- 172.21.15.106:0/1280872664 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4001850 con 0x7fd4c0057620 2024-04-25T11:19:35.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f062031a640 1 -- start start 2024-04-25T11:19:35.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7ff5cdedf640 1 -- 172.21.15.106:0/989921716 >> v1:172.21.15.134:6789/0 conn(0x7ff5c8072ac0 legacy=0x7ff5c8072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227d096640 1 -- 172.21.15.106:0/637263274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 msgr2=0x7f227806fa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:19:35.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227d096640 1 --2- 172.21.15.106:0/637263274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 0x7f227806fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227d096640 1 -- 172.21.15.106:0/637263274 >> v2:172.21.15.134:3300/0 conn(0x7f22780b2de0 msgr2=0x7f22780b31f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:19:35.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227d096640 1 --2- 172.21.15.106:0/637263274 >> v2:172.21.15.134:3300/0 conn(0x7f22780b2de0 0x7f22780b31f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.253+0000 7f227d096640 1 -- 172.21.15.106:0/637263274 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2278160f20 con 0x7f2278160750 2024-04-25T11:19:35.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f227d096640 1 --2- 172.21.15.106:0/637263274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2278160750 0x7f227806ff40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2278072620 tx=0x7f226801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299319b6caae8032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7ff5cdedf640 1 -- 172.21.15.106:0/989921716 >> v1:172.21.15.106:6789/0 conn(0x7ff5c815ce30 legacy=0x7ff5c8152b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:19:35.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7ff5cdedf640 1 -- 172.21.15.106:0/989921716 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80b48c0 con 0x7ff5c80b6030 2024-04-25T11:19:35.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f227c895640 1 -- 172.21.15.106:0/637263274 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22680b0070 con 0x7f2278160750 2024-04-25T11:19:35.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f227c895640 1 -- 172.21.15.106:0/637263274 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22680ab600 con 0x7f2278160750 2024-04-25T11:19:35.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f227c895640 1 -- 172.21.15.106:0/637263274 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22680ab9d0 con 0x7f2278160750 2024-04-25T11:19:35.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1280872664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 msgr2=0x7fd4c0057da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4cd0c3640 1 --2- 172.21.15.106:0/1280872664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c0057da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097f0 tx=0x7fd4bc01c3c0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4983fff640 1 --2- 172.21.15.106:0/1546334456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49840b23a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4984069330 tx=0x7f497c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2f0102d31c8b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90f55c0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80e5850 0x7f90e80062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90f55c0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e80515a0 con 0x7f90e80340f0 2024-04-25T11:19:35.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90f55c0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e8034500 con 0x7f90e80e5c60 2024-04-25T11:19:35.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90f55c0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f90e80548f0 con 0x7f90e80e5850 2024-04-25T11:19:35.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1280872664 shutdown_connections 2024-04-25T11:19:35.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4cd0c3640 1 --2- 172.21.15.106:0/1280872664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c0057da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1280872664 >> 172.21.15.106:0/1280872664 conn(0x7fd4c003ac60 msgr2=0x7fd4c003b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90effff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f90e80340f0 0x7f90e80e5140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57092/0 (socket says 172.21.15.106:57092) 2024-04-25T11:19:35.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90effff640 1 -- 172.21.15.106:0/875694951 learned_addr learned my addr 172.21.15.106:0/875694951 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:35.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f49827fc640 1 -- 172.21.15.106:0/1546334456 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c0b2070 con 0x7f49840b1f90 2024-04-25T11:19:35.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f49827fc640 1 -- 172.21.15.106:0/1546334456 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c0a5920 con 0x7f49840b1f90 2024-04-25T11:19:35.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f54fbc5b640 1 Processor -- start 2024-04-25T11:19:35.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f227fafb640 1 -- 172.21.15.106:0/637263274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2278160750 msgr2=0x7f227806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f227fafb640 1 --2- 172.21.15.106:0/637263274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2278160750 0x7f227806ff40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2278072620 tx=0x7f226801c560 comp rx=0 tx=0).stop 2024-04-25T11:19:35.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1280872664 shutdown_connections 2024-04-25T11:19:35.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1280872664 wait complete. 2024-04-25T11:19:35.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f49827fc640 1 -- 172.21.15.106:0/1546334456 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c0a5d30 con 0x7f49840b1f90 2024-04-25T11:19:35.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4989c29640 1 -- 172.21.15.106:0/1546334456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 msgr2=0x7f49840b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4989c29640 1 --2- 172.21.15.106:0/1546334456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49840b23a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4984069330 tx=0x7f497c01c970 comp rx=0 tx=0).stop 2024-04-25T11:19:35.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4989c29640 1 -- 172.21.15.106:0/1546334456 shutdown_connections 2024-04-25T11:19:35.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4989c29640 1 --2- 172.21.15.106:0/1546334456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49840b23a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4989c29640 1 --2- 172.21.15.106:0/1546334456 >> v2:172.21.15.134:3300/0 conn(0x7f49840b4830 0x7f49840b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4989c29640 1 -- 172.21.15.106:0/1546334456 >> 172.21.15.106:0/1546334456 conn(0x7f498409ba30 msgr2=0x7f49840b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90ef7fe640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80e5850 0x7f90e80062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2095685009 0 0) 0x7f90e8034500 con 0x7f90e80e5c60 2024-04-25T11:19:35.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90e0003530 con 0x7f90e80e5c60 2024-04-25T11:19:35.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4989c29640 1 -- 172.21.15.106:0/1546334456 shutdown_connections 2024-04-25T11:19:35.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f4989c29640 1 -- 172.21.15.106:0/1546334456 wait complete. 2024-04-25T11:19:35.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828423931 0 0) 0x7f90dc001120 con 0x7f90e80340f0 2024-04-25T11:19:35.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f54fbc5b640 1 -- start start 2024-04-25T11:19:35.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f227fafb640 1 -- 172.21.15.106:0/637263274 shutdown_connections 2024-04-25T11:19:35.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f227fafb640 1 --2- 172.21.15.106:0/637263274 >> v2:172.21.15.134:3300/0 conn(0x7f22780b2de0 0x7f22780b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f227fafb640 1 --2- 172.21.15.106:0/637263274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2278160750 0x7f227806ff40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f227fafb640 1 --2- 172.21.15.106:0/637263274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 0x7f227806fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f227fafb640 1 -- 172.21.15.106:0/637263274 >> 172.21.15.106:0/637263274 conn(0x7f227806cbf0 msgr2=0x7f22780d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.257+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90e8034500 con 0x7f90e80340f0 2024-04-25T11:19:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 736430210 0 0) 0x7f90e0003530 con 0x7f90e80e5c60 2024-04-25T11:19:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80e5850 msgr2=0x7f90e80062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80e5850 0x7f90e80062a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 >> v1:172.21.15.106:6789/0 conn(0x7f90e80340f0 legacy=0x7f90e80e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e8033a80 con 0x7f90e80e5c60 2024-04-25T11:19:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f90f006aae0 con 0x7f90e80e5c60 2024-04-25T11:19:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 >> v1:172.21.15.134:6789/0 conn(0x7f90e80e5c60 legacy=0x7f90e80067e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e0003b50 0x7f90e0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00079b0 0x7f90e0009dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e000a310 0x7f90e000c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f90e80548f0 con 0x7f90e000a310 2024-04-25T11:19:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90e000cfa0 con 0x7f90e00079b0 2024-04-25T11:19:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f90e000d1a0 con 0x7f90e0003b50 2024-04-25T11:19:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90f4dbf640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e000a310 0x7f90e000c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90ef7fe640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00079b0 0x7f90e0009dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54fbc5b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f54f406d110 0x7f54f40b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54fbc5b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f406d520 con 0x7f54f40b3360 2024-04-25T11:19:35.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5cdedf640 1 --2- 172.21.15.106:0/989921716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c80b6030 0x7ff5c80b50c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80097c0 tx=0x7ff5b801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7712c5c906d854e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5ccedd640 1 -- 172.21.15.106:0/989921716 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80b0070 con 0x7ff5c80b6030 2024-04-25T11:19:35.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5ccedd640 1 -- 172.21.15.106:0/989921716 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b80ab600 con 0x7ff5c80b6030 2024-04-25T11:19:35.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5ccedd640 1 -- 172.21.15.106:0/989921716 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80ab9d0 con 0x7ff5c80b6030 2024-04-25T11:19:35.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7fd4cd0c3640 1 Processor -- start 2024-04-25T11:19:35.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f227fafb640 1 -- 172.21.15.106:0/637263274 shutdown_connections 2024-04-25T11:19:35.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7fd4cd0c3640 1 -- start start 2024-04-25T11:19:35.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90f4dbf640 1 -- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e0003b50 msgr2=0x7f90e0003fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:19:35.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90f4dbf640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e0003b50 0x7f90e0003fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90f4dbf640 1 -- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00079b0 msgr2=0x7f90e0009dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90f4dbf640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00079b0 0x7f90e0009dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90f4dbf640 1 -- 172.21.15.106:0/875694951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e8033a80 con 0x7f90e000a310 2024-04-25T11:19:35.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f4989c29640 1 Processor -- start 2024-04-25T11:19:35.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f4989c29640 1 -- start start 2024-04-25T11:19:35.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f227fafb640 1 -- 172.21.15.106:0/637263274 wait complete. 2024-04-25T11:19:35.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54fbc5b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f40cf040 con 0x7f54f406c5f0 2024-04-25T11:19:35.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54fbc5b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f54f40d1160 con 0x7f54f406d110 2024-04-25T11:19:35.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f99f7640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f54f40b3360 0x7f54f40d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57098/0 (socket says 172.21.15.106:57098) 2024-04-25T11:19:35.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f99f7640 1 -- 172.21.15.106:0/2298854897 learned_addr learned my addr 172.21.15.106:0/2298854897 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:35.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5d0143640 1 -- 172.21.15.106:0/989921716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c80b6030 msgr2=0x7ff5c80b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5d0143640 1 --2- 172.21.15.106:0/989921716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c80b6030 0x7ff5c80b50c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80097c0 tx=0x7ff5b801c480 comp rx=0 tx=0).stop 2024-04-25T11:19:35.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54fa1f8640 1 --2- 172.21.15.106:0/2298854897 >> v2:172.21.15.134:3300/0 conn(0x7f54f406d110 0x7f54f40b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2799873855 0 0) 0x7f54ec0014a0 con 0x7f54f40b3360 2024-04-25T11:19:35.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54e0003530 con 0x7f54f40b3360 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 241103750 0 0) 0x7f54e0003530 con 0x7f54f40b3360 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 >> v2:172.21.15.134:3300/0 conn(0x7f54f406d110 msgr2=0x7f54f40b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 --2- 172.21.15.106:0/2298854897 >> v2:172.21.15.134:3300/0 conn(0x7f54f406d110 0x7f54f40b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 >> v1:172.21.15.134:6789/0 conn(0x7f54f406c5f0 legacy=0x7f54f406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f40cf320 con 0x7f54f40b3360 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f54ec001bd0 con 0x7f54f40b3360 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 >> v1:172.21.15.106:6789/0 conn(0x7f54f40b3360 legacy=0x7f54f40d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e00039c0 0x7f54e0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e0004810 0x7f54e0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e0005280 0x7f54e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54e000c1e0 con 0x7f54e00039c0 2024-04-25T11:19:35.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54e000c390 con 0x7f54e0004810 2024-04-25T11:19:35.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f54e000c590 con 0x7f54e0005280 2024-04-25T11:19:35.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5d0143640 1 -- 172.21.15.106:0/989921716 shutdown_connections 2024-04-25T11:19:35.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5d0143640 1 --2- 172.21.15.106:0/989921716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c80b6030 0x7ff5c80b50c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5d0143640 1 -- 172.21.15.106:0/989921716 >> 172.21.15.106:0/989921716 conn(0x7ff5c80d13f0 msgr2=0x7ff5c80d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5d0143640 1 -- 172.21.15.106:0/989921716 shutdown_connections 2024-04-25T11:19:35.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7ff5d0143640 1 -- 172.21.15.106:0/989921716 wait complete. 2024-04-25T11:19:35.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f99f7640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e00039c0 0x7f54e0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f91f6640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e0004810 0x7f54e0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54fa1f8640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e0005280 0x7f54e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f99f7640 1 -- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e0005280 msgr2=0x7f54e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f99f7640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e0005280 0x7f54e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f99f7640 1 -- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e0004810 msgr2=0x7f54e0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f99f7640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e0004810 0x7f54e0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54f99f7640 1 -- 172.21.15.106:0/2298854897 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f40cf320 con 0x7f54e00039c0 2024-04-25T11:19:35.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f90f4dbf640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e000a310 0x7f90e000c730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90f006afe0 tx=0x7f90f00996f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b2f72bc83a0b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7fd4cd0c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4c003e430 0x7fd4c00d1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.261+0000 7f54fa1f8640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e0005280 0x7f54e000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:35.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f54fa1f8640 1 -- 172.21.15.106:0/2298854897 reap_dead start 2024-04-25T11:19:35.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4cd0c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c00d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4989c29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49841fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4cd0c3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c00582e0 0x7fd4c00d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4cd0c3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c003c9e0 con 0x7fd4c00582e0 2024-04-25T11:19:35.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4c003e430 0x7fd4c00d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4c003e430 0x7fd4c00d1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45440/0 (socket says 172.21.15.106:45440) 2024-04-25T11:19:35.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c7fff640 1 -- 172.21.15.106:0/1515510506 learned_addr learned my addr 172.21.15.106:0/1515510506 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c77fe640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c00d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c003c860 con 0x7fd4c003e430 2024-04-25T11:19:35.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c00f1c20 con 0x7fd4c0057620 2024-04-25T11:19:35.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49841fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49841fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46438/0 (socket says 172.21.15.106:46438) 2024-04-25T11:19:35.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 -- 172.21.15.106:0/2237175706 learned_addr learned my addr 172.21.15.106:0/2237175706 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227fafb640 1 Processor -- start 2024-04-25T11:19:35.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4989c29640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49840b4830 0x7f4984201020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4989c29640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49841605e0 0x7f4984201560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4989c29640 1 -- 172.21.15.106:0/2237175706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49840b4dc0 con 0x7f49840b1f90 2024-04-25T11:19:35.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f062031a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f06180d1fa0 0x7f06180d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c77fe640 1 -- 172.21.15.106:0/1515510506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4c003e430 msgr2=0x7fd4c00d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c77fe640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4c003e430 0x7fd4c00d1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c77fe640 1 -- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c00582e0 msgr2=0x7fd4c00d93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c77fe640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c00582e0 0x7fd4c00d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0076ca0 con 0x7f90e000a310 2024-04-25T11:19:35.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f0127700 con 0x7f90e000a310 2024-04-25T11:19:35.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90eeffd640 1 -- 172.21.15.106:0/875694951 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0123040 con 0x7f90e000a310 2024-04-25T11:19:35.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90f55c0640 1 -- 172.21.15.106:0/875694951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e000a310 msgr2=0x7f90e000c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90f55c0640 1 --2- 172.21.15.106:0/875694951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e000a310 0x7f90e000c730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90f006afe0 tx=0x7f90f00996f0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c77fe640 1 -- 172.21.15.106:0/1515510506 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c00c91a0 con 0x7fd4c0057620 2024-04-25T11:19:35.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4982ffd640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49840b4830 0x7f4984201020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4983fff640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49841605e0 0x7f4984201560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f062031a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06180d23b0 0x7f06180b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f062031a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06180c6cf0 con 0x7f06180d17b0 2024-04-25T11:19:35.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f062031a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f06180c6aa0 con 0x7f06180d23b0 2024-04-25T11:19:35.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f062031a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f06180b8490 con 0x7f06180d1fa0 2024-04-25T11:19:35.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f061e0b6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f06180d1fa0 0x7f06180d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f061e0b6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f06180d1fa0 0x7f06180d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45456/0 (socket says 172.21.15.106:45456) 2024-04-25T11:19:35.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f061e8b7640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f06180d17b0 0x7f06180b67a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57114/0 (socket says 172.21.15.106:57114) 2024-04-25T11:19:35.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4989c29640 1 -- 172.21.15.106:0/2237175706 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49840b4c40 con 0x7f49840b4830 2024-04-25T11:19:35.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f061e0b6640 1 -- 172.21.15.106:0/3115388220 learned_addr learned my addr 172.21.15.106:0/3115388220 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4989c29640 1 -- 172.21.15.106:0/2237175706 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49840b28e0 con 0x7f49841605e0 2024-04-25T11:19:35.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90f4dbf640 1 -- 172.21.15.106:0/875694951 reap_dead start 2024-04-25T11:19:35.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c77fe640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c00d8e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c4b0 tx=0x7fd4b400c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90f55c0640 1 -- 172.21.15.106:0/875694951 shutdown_connections 2024-04-25T11:19:35.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90f55c0640 1 -- 172.21.15.106:0/875694951 >> 172.21.15.106:0/875694951 conn(0x7f90e8054d00 msgr2=0x7f90e8055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90f55c0640 1 -- 172.21.15.106:0/875694951 shutdown_connections 2024-04-25T11:19:35.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c4ff9640 1 -- 172.21.15.106:0/1515510506 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4018070 con 0x7fd4c0057620 2024-04-25T11:19:35.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 -- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49841605e0 msgr2=0x7f4984201560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49841605e0 0x7f4984201560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 -- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49840b4830 msgr2=0x7f4984201020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49840b4830 0x7f4984201020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 -- 172.21.15.106:0/2237175706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4984152100 con 0x7f49840b1f90 2024-04-25T11:19:35.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227fafb640 1 -- start start 2024-04-25T11:19:35.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227fafb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22780b2de0 0x7f227814cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227fafb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 0x7f22780c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f49837fe640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49841fe5e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f497400b3e0 tx=0x7f497400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227fafb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2278160750 0x7f22780732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f54f99f7640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e00039c0 0x7f54e0003e30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00aa70 tx=0x7f54ec01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53b935b1fb3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227d897640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22780b2de0 0x7f227814cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227d897640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22780b2de0 0x7f227814cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45472/0 (socket says 172.21.15.106:45472) 2024-04-25T11:19:35.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227d897640 1 -- 172.21.15.106:0/1397931450 learned_addr learned my addr 172.21.15.106:0/1397931450 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227d096640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 0x7f22780c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f227806f280 con 0x7f2278160750 2024-04-25T11:19:35.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f227806f100 con 0x7f22780b2de0 2024-04-25T11:19:35.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f227806f400 con 0x7f2278160340 2024-04-25T11:19:35.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c4ff9640 1 -- 172.21.15.106:0/1515510506 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b40074e0 con 0x7fd4c0057620 2024-04-25T11:19:35.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4c4ff9640 1 -- 172.21.15.106:0/1515510506 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4004a30 con 0x7fd4c0057620 2024-04-25T11:19:35.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f496bfff640 1 -- 172.21.15.106:0/2237175706 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974015020 con 0x7f49840b1f90 2024-04-25T11:19:35.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f496bfff640 1 -- 172.21.15.106:0/2237175706 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4974011830 con 0x7f49840b1f90 2024-04-25T11:19:35.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f496bfff640 1 -- 172.21.15.106:0/2237175706 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974011c00 con 0x7f49840b1f90 2024-04-25T11:19:35.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4989c29640 1 -- 172.21.15.106:0/2237175706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498415b320 con 0x7f49840b1f90 2024-04-25T11:19:35.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f4989c29640 1 -- 172.21.15.106:0/2237175706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498415b8e0 con 0x7f49840b1f90 2024-04-25T11:19:35.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c00d4920 con 0x7fd4c0057620 2024-04-25T11:19:35.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c00d9a60 con 0x7fd4c0057620 2024-04-25T11:19:35.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f4989c29640 1 -- 172.21.15.106:0/2237175706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4984050750 con 0x7f49840b1f90 2024-04-25T11:19:35.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f061e0b6640 1 -- 172.21.15.106:0/3115388220 >> v2:172.21.15.106:3300/0 conn(0x7f06180d23b0 msgr2=0x7f06180b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f061e0b6640 1 --2- 172.21.15.106:0/3115388220 >> v2:172.21.15.106:3300/0 conn(0x7f06180d23b0 0x7f06180b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7ff5d0143640 1 Processor -- start 2024-04-25T11:19:35.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f061e0b6640 1 -- 172.21.15.106:0/3115388220 >> v1:172.21.15.106:6789/0 conn(0x7f06180d17b0 legacy=0x7f06180b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f061e0b6640 1 -- 172.21.15.106:0/3115388220 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06180b8be0 con 0x7f06180d1fa0 2024-04-25T11:19:35.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7ff5d0143640 1 -- start start 2024-04-25T11:19:35.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f227d897640 1 -- 172.21.15.106:0/1397931450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 msgr2=0x7f22780c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f227d897640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 0x7f22780c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f227d897640 1 -- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2278160750 msgr2=0x7f22780732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f227d897640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2278160750 0x7f22780732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f227d897640 1 -- 172.21.15.106:0/1397931450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227814d290 con 0x7f22780b2de0 2024-04-25T11:19:35.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00a670 con 0x7f54e00039c0 2024-04-25T11:19:35.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ec0072d0 con 0x7f54e00039c0 2024-04-25T11:19:35.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f54f89f5640 1 -- 172.21.15.106:0/2298854897 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec007700 con 0x7f54e00039c0 2024-04-25T11:19:35.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.265+0000 7f90f55c0640 1 -- 172.21.15.106:0/875694951 wait complete. 2024-04-25T11:19:35.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f90f55c0640 1 Processor -- start 2024-04-25T11:19:35.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f061e0b6640 1 --2- 172.21.15.106:0/3115388220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d1fa0 0x7f06180d1270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0618073b00 tx=0x7f061001c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1cbd6cbf3d9e2a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f227d897640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22780b2de0 0x7f227814cd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f227400b440 tx=0x7f227400b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c0004870 con 0x7fd4c0057620 2024-04-25T11:19:35.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2298854897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e00039c0 msgr2=0x7f54e0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e00039c0 0x7f54e0003e30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00aa70 tx=0x7f54ec01d340 comp rx=0 tx=0).stop 2024-04-25T11:19:35.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f226e7fc640 1 -- 172.21.15.106:0/1397931450 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274015020 con 0x7f22780b2de0 2024-04-25T11:19:35.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f90f55c0640 1 -- start start 2024-04-25T11:19:35.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f061d0b4640 1 -- 172.21.15.106:0/3115388220 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06100a57d0 con 0x7f06180d1fa0 2024-04-25T11:19:35.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f061d0b4640 1 -- 172.21.15.106:0/3115388220 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06100a5970 con 0x7f06180d1fa0 2024-04-25T11:19:35.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f061d0b4640 1 -- 172.21.15.106:0/3115388220 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06100a5d80 con 0x7f06180d1fa0 2024-04-25T11:19:35.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227815c910 con 0x7f22780b2de0 2024-04-25T11:19:35.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2278072aa0 con 0x7f22780b2de0 2024-04-25T11:19:35.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f062031a640 1 -- 172.21.15.106:0/3115388220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d1fa0 msgr2=0x7f06180d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f062031a640 1 --2- 172.21.15.106:0/3115388220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d1fa0 0x7f06180d1270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0618073b00 tx=0x7f061001c910 comp rx=0 tx=0).stop 2024-04-25T11:19:35.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f496bfff640 1 -- 172.21.15.106:0/2237175706 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4974018020 con 0x7f49840b1f90 2024-04-25T11:19:35.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f496bfff640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f497006f500 0x7f4970071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f496bfff640 1 -- 172.21.15.106:0/2237175706 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f497408f330 con 0x7f49840b1f90 2024-04-25T11:19:35.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2298854897 shutdown_connections 2024-04-25T11:19:35.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2298854897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e00039c0 0x7f54e0003e30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2298854897 >> 172.21.15.106:0/2298854897 conn(0x7f54f40b4020 msgr2=0x7f54f40b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f062031a640 1 -- 172.21.15.106:0/3115388220 shutdown_connections 2024-04-25T11:19:35.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f062031a640 1 --2- 172.21.15.106:0/3115388220 >> v2:172.21.15.106:3300/0 conn(0x7f06180d23b0 0x7f06180b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f062031a640 1 --2- 172.21.15.106:0/3115388220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d1fa0 0x7f06180d1270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f062031a640 1 -- 172.21.15.106:0/3115388220 >> 172.21.15.106:0/3115388220 conn(0x7f06180cfe90 msgr2=0x7f06180d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f90f55c0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e80cb010 0x7f90e80e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7ff5d0143640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5c8072ac0 0x7ff5c8143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7ff5d0143640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c80b6030 0x7ff5c8144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f90f55c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e80dd440 0x7f90e80d8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5d0143640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c815ce30 0x7ff5c815e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5d0143640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c806f690 con 0x7ff5c8072ac0 2024-04-25T11:19:35.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5d0143640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c806f510 con 0x7ff5c815ce30 2024-04-25T11:19:35.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5d0143640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c806f810 con 0x7ff5c80b6030 2024-04-25T11:19:35.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5c8072ac0 0x7ff5c8143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5c8072ac0 0x7ff5c8143f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46480/0 (socket says 172.21.15.106:46480) 2024-04-25T11:19:35.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 -- 172.21.15.106:0/736954099 learned_addr learned my addr 172.21.15.106:0/736954099 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2298854897 shutdown_connections 2024-04-25T11:19:35.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90effff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e80cb010 0x7f90e80e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90effff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e80cb010 0x7f90e80e2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46496/0 (socket says 172.21.15.106:46496) 2024-04-25T11:19:35.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f062031a640 1 -- 172.21.15.106:0/3115388220 shutdown_connections 2024-04-25T11:19:35.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cd6de640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c80b6030 0x7ff5c8144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5ce6e0640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c815ce30 0x7ff5c815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f062031a640 1 -- 172.21.15.106:0/3115388220 wait complete. 2024-04-25T11:19:35.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f226e7fc640 1 -- 172.21.15.106:0/1397931450 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2274011890 con 0x7f22780b2de0 2024-04-25T11:19:35.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.269+0000 7f226e7fc640 1 -- 172.21.15.106:0/1397931450 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274011c60 con 0x7f22780b2de0 2024-04-25T11:19:35.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f4982ffd640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f497006f500 0x7f4970071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f55c0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80dd850 0x7f90e80d8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f55c0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f90e803ab50 con 0x7f90e80cb010 2024-04-25T11:19:35.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f55c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90e803a870 con 0x7f90e80dd440 2024-04-25T11:19:35.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f55c0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f90e803a520 con 0x7f90e80dd850 2024-04-25T11:19:35.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7fd4c4ff9640 1 -- 172.21.15.106:0/1515510506 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4b4004bd0 con 0x7fd4c0057620 2024-04-25T11:19:35.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7fd4c4ff9640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4ac06f400 0x7fd4ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7fd4c4ff9640 1 -- 172.21.15.106:0/1515510506 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd4b408e4b0 con 0x7fd4c0057620 2024-04-25T11:19:35.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 -- 172.21.15.106:0/736954099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c80b6030 msgr2=0x7ff5c8144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c80b6030 0x7ff5c8144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 -- 172.21.15.106:0/736954099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c815ce30 msgr2=0x7ff5c815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c815ce30 0x7ff5c815e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 -- 172.21.15.106:0/736954099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c8144470 con 0x7ff5c8072ac0 2024-04-25T11:19:35.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7ff5cdedf640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5c8072ac0 0x7ff5c8143f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8019090 tx=0x7ff5b8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f4982ffd640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f497006f500 0x7f4970071a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4978005890 tx=0x7f4978009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7fd4c7fff640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4ac06f400 0x7fd4ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80dd850 0x7f90e80d8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80dd850 0x7f90e80d8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40384/0 (socket says 172.21.15.106:40384) 2024-04-25T11:19:35.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 -- 172.21.15.106:0/2327245632 learned_addr learned my addr 172.21.15.106:0/2327245632 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7fd4c7fff640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4ac06f400 0x7fd4ac071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097f0 tx=0x7fd4bc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2298854897 wait complete. 2024-04-25T11:19:35.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f226e7fc640 1 -- 172.21.15.106:0/1397931450 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2274018020 con 0x7f22780b2de0 2024-04-25T11:19:35.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22600052a0 con 0x7f22780b2de0 2024-04-25T11:19:35.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 -- 172.21.15.106:0/2327245632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e80dd440 msgr2=0x7f90e80d8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e80dd440 0x7f90e80d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 -- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e80cb010 msgr2=0x7f90e80e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e80cb010 0x7f90e80e2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 -- 172.21.15.106:0/2327245632 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e80e35a0 con 0x7f90e80dd850 2024-04-25T11:19:35.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f226e7fc640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f226406f590 0x7f2264071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f226e7fc640 1 -- 172.21.15.106:0/1397931450 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f227408f2f0 con 0x7f22780b2de0 2024-04-25T11:19:35.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f062031a640 1 Processor -- start 2024-04-25T11:19:35.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f062031a640 1 -- start start 2024-04-25T11:19:35.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f227d096640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f226406f590 0x7f2264071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f90f4dbf640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80dd850 0x7f90e80d8960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90f0065bf0 tx=0x7f90f0096530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f54fbc5b640 1 Processor -- start 2024-04-25T11:19:35.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f227d096640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f226406f590 0x7f2264071aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22680ad000 tx=0x7f22680a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.273+0000 7f54fbc5b640 1 -- start start 2024-04-25T11:19:35.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f062031a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d17b0 0x7f061809d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f062031a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06180d1fa0 0x7f0618205f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f062031a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06180d23b0 0x7f06181fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f062031a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06180ce560 con 0x7f06180d23b0 2024-04-25T11:19:35.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f062031a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06180ce3e0 con 0x7f06180d17b0 2024-04-25T11:19:35.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f062031a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06180ce6e0 con 0x7f06180d1fa0 2024-04-25T11:19:35.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06180d23b0 0x7f06181fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06180d23b0 0x7f06181fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46502/0 (socket says 172.21.15.106:46502) 2024-04-25T11:19:35.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 -- 172.21.15.106:0/3009980143 learned_addr learned my addr 172.21.15.106:0/3009980143 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061d8b5640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06180d1fa0 0x7f0618205f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e0b6640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d17b0 0x7f061809d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7ff5be7fc640 1 -- 172.21.15.106:0/736954099 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80b0040 con 0x7ff5c8072ac0 2024-04-25T11:19:35.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f496bfff640 1 -- 172.21.15.106:0/2237175706 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f497401f070 con 0x7f49840b1f90 2024-04-25T11:19:35.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c8145140 con 0x7ff5c8072ac0 2024-04-25T11:19:35.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 -- 172.21.15.106:0/3009980143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06180d1fa0 msgr2=0x7f0618205f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06180d1fa0 0x7f0618205f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 -- 172.21.15.106:0/3009980143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d17b0 msgr2=0x7f061809d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d17b0 0x7f061809d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 -- 172.21.15.106:0/3009980143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061809fe60 con 0x7f06180d23b0 2024-04-25T11:19:35.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f061e8b7640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06180d23b0 0x7f06181fb8a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f061400c3c0 tx=0x7f061400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fbc5b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00039c0 0x7f54f420a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fbc5b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f40cab90 0x7f54f414c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54f99f7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00039c0 0x7f54f420a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54f99f7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00039c0 0x7f54f420a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:40396/0 (socket says 172.21.15.106:40396) 2024-04-25T11:19:35.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54f99f7640 1 -- 172.21.15.106:0/2054618215 learned_addr learned my addr 172.21.15.106:0/2054618215 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c8145700 con 0x7ff5c8072ac0 2024-04-25T11:19:35.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7ff5be7fc640 1 -- 172.21.15.106:0/736954099 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8000e50 con 0x7ff5c8072ac0 2024-04-25T11:19:35.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f40cafa0 0x7f54f414c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7ff5be7fc640 1 -- 172.21.15.106:0/736954099 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80b5680 con 0x7ff5c8072ac0 2024-04-25T11:19:35.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54f91f6640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f40cab90 0x7f54f414c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40ce040 con 0x7f54f40cafa0 2024-04-25T11:19:35.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40cfc50 con 0x7f54f40cab90 2024-04-25T11:19:35.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f54f40ce1c0 con 0x7f54e00039c0 2024-04-25T11:19:35.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b00052a0 con 0x7ff5c8072ac0 2024-04-25T11:19:35.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fa1f8640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f40cafa0 0x7f54f414c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fa1f8640 1 -- 172.21.15.106:0/2054618215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00039c0 msgr2=0x7f54f420a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fa1f8640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00039c0 0x7f54f420a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fa1f8640 1 -- 172.21.15.106:0/2054618215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f40cab90 msgr2=0x7f54f414c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fa1f8640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f40cab90 0x7f54f414c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fa1f8640 1 -- 172.21.15.106:0/2054618215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f40cb760 con 0x7f54f40cafa0 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54fa1f8640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f40cafa0 0x7f54f414c950 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f54f40d3570 tx=0x7f54f000e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f54ea7fc640 1 -- 172.21.15.106:0/2054618215 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f000f040 con 0x7f54f40cafa0 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f4206f50 con 0x7f54f40cafa0 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.277+0000 7f060effd640 1 -- 172.21.15.106:0/3009980143 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06140074e0 con 0x7f06180d23b0 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0618206480 con 0x7f06180d23b0 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0618206c70 con 0x7f06180d23b0 2024-04-25T11:19:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f060effd640 1 -- 172.21.15.106:0/3009980143 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0614007680 con 0x7f06180d23b0 2024-04-25T11:19:35.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f060effd640 1 -- 172.21.15.106:0/3009980143 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614004a10 con 0x7f06180d23b0 2024-04-25T11:19:35.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f54ea7fc640 1 -- 172.21.15.106:0/2054618215 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f0019070 con 0x7f54f40cafa0 2024-04-25T11:19:35.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f54ea7fc640 1 -- 172.21.15.106:0/2054618215 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0013830 con 0x7f54f40cafa0 2024-04-25T11:19:35.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f54ea7fc640 1 -- 172.21.15.106:0/2054618215 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f54f00139d0 con 0x7f54f40cafa0 2024-04-25T11:19:35.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f4209c50 con 0x7f54f40cafa0 2024-04-25T11:19:35.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f54ea7fc640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f54d806f400 0x7f54d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f4050750 con 0x7f54f40cafa0 2024-04-25T11:19:35.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7f54f99f7640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f54d806f400 0x7f54d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.285+0000 7f54f99f7640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f54d806f400 0x7f54d8071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00a0d0 tx=0x7f54ec0af040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7ff5be7fc640 1 -- 172.21.15.106:0/736954099 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff5b80b3070 con 0x7ff5c8072ac0 2024-04-25T11:19:35.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7ff5be7fc640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff5ac06f430 0x7ff5ac071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7ff5be7fc640 1 -- 172.21.15.106:0/736954099 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5b8128b90 con 0x7ff5c8072ac0 2024-04-25T11:19:35.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7ff5cd6de640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff5ac06f430 0x7ff5ac071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.281+0000 7ff5cd6de640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff5ac06f430 0x7ff5ac071940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8152910 tx=0x7ff5c4009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.285+0000 7ff5be7fc640 1 -- 172.21.15.106:0/736954099 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff5b80c5020 con 0x7ff5c8072ac0 2024-04-25T11:19:35.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.285+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06000052a0 con 0x7f06180d23b0 2024-04-25T11:19:35.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f54ea7fc640 1 -- 172.21.15.106:0/2054618215 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f54f008c750 con 0x7f54f40cafa0 2024-04-25T11:19:35.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90ecff9640 1 -- 172.21.15.106:0/2327245632 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f006aae0 con 0x7f90e80dd850 2024-04-25T11:19:35.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7fd4c4ff9640 1 -- 172.21.15.106:0/1515510506 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4c00586f0 con 0x7fd4c0057620 2024-04-25T11:19:35.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e80e3880 con 0x7f90e80dd850 2024-04-25T11:19:35.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e80e3e40 con 0x7f90e80dd850 2024-04-25T11:19:35.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90ecff9640 1 -- 172.21.15.106:0/2327245632 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f0069020 con 0x7f90e80dd850 2024-04-25T11:19:35.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90ecff9640 1 -- 172.21.15.106:0/2327245632 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0127700 con 0x7f90e80dd850 2024-04-25T11:19:35.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f060effd640 1 -- 172.21.15.106:0/3009980143 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f061401a020 con 0x7f06180d23b0 2024-04-25T11:19:35.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f060effd640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05fc06f400 0x7f05fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f060effd640 1 -- 172.21.15.106:0/3009980143 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f061408f050 con 0x7f06180d23b0 2024-04-25T11:19:35.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f061e0b6640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05fc06f400 0x7f05fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d00052a0 con 0x7f90e80dd850 2024-04-25T11:19:35.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f061e0b6640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05fc06f400 0x7f05fc071910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f06100ae000 tx=0x7f06100a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f226e7fc640 1 -- 172.21.15.106:0/1397931450 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f227401f020 con 0x7f22780b2de0 2024-04-25T11:19:35.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90ecff9640 1 -- 172.21.15.106:0/2327245632 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f90f00691d0 con 0x7f90e80dd850 2024-04-25T11:19:35.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90ecff9640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f90d406f5a0 0x7f90d4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.289+0000 7f90ecff9640 1 -- 172.21.15.106:0/2327245632 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f90f01a4f40 con 0x7f90e80dd850 2024-04-25T11:19:35.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.293+0000 7f90effff640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f90d406f5a0 0x7f90d4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.293+0000 7f90effff640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f90d406f5a0 0x7f90d4071ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90dc001060 tx=0x7f90dc0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.297+0000 7f54ea7fc640 1 -- 172.21.15.106:0/2054618215 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f54f005a930 con 0x7f54f40cafa0 2024-04-25T11:19:35.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.305+0000 7f90ecff9640 1 -- 172.21.15.106:0/2327245632 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f90f0093640 con 0x7f90e80dd850 2024-04-25T11:19:35.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.305+0000 7f060effd640 1 -- 172.21.15.106:0/3009980143 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0614021070 con 0x7f06180d23b0 2024-04-25T11:19:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.561+0000 7f0eac309640 1 -- 172.21.15.106:0/1952552021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0e8c005590 con 0x7f0ea409ee40 2024-04-25T11:19:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e9affd640 1 -- 172.21.15.106:0/1952552021 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e9c05b830 con 0x7f0ea409ee40 2024-04-25T11:19:35.565 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T11:19:35.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 -- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0e8806f4d0 msgr2=0x7f0e880719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0e8806f4d0 0x7f0e880719e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00ad000 tx=0x7f0ea00a6000 comp rx=0 tx=0).stop 2024-04-25T11:19:35.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 -- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ea409ee40 msgr2=0x7f0ea40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ea409ee40 0x7f0ea40b4f30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c760 tx=0x7f0e9c00cc20 comp rx=0 tx=0).stop 2024-04-25T11:19:35.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 -- 172.21.15.106:0/1952552021 shutdown_connections 2024-04-25T11:19:35.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0e8806f4d0 0x7f0e880719e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ea409ee40 0x7f0ea40b4f30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea40cbbd0 0x7f0ea409e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 --2- 172.21.15.106:0/1952552021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ea40cb7c0 0x7f0ea40b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.565+0000 7f0e98ef9640 1 -- 172.21.15.106:0/1952552021 >> 172.21.15.106:0/1952552021 conn(0x7f0ea40cdc80 msgr2=0x7f0ea40ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.569+0000 7f0e98ef9640 1 -- 172.21.15.106:0/1952552021 shutdown_connections 2024-04-25T11:19:35.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.569+0000 7f0e98ef9640 1 -- 172.21.15.106:0/1952552021 wait complete. 2024-04-25T11:19:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.585+0000 7f4989c29640 1 -- 172.21.15.106:0/2237175706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f49840b6430 con 0x7f49840b1f90 2024-04-25T11:19:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.585+0000 7f496bfff640 1 -- 172.21.15.106:0/2237175706 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f497405bbb0 con 0x7f49840b1f90 2024-04-25T11:19:35.583 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T11:19:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.589+0000 7f4969ffb640 1 -- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f497006f500 msgr2=0x7f4970071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.589+0000 7f4969ffb640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f497006f500 0x7f4970071a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4978005890 tx=0x7f4978009040 comp rx=0 tx=0).stop 2024-04-25T11:19:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.589+0000 7f4969ffb640 1 -- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 msgr2=0x7f49841fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.589+0000 7f4969ffb640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49841fe5e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f497400b3e0 tx=0x7f497400b8a0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.589+0000 7f4969ffb640 1 -- 172.21.15.106:0/2237175706 shutdown_connections 2024-04-25T11:19:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.593+0000 7f4969ffb640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49841605e0 0x7f4984201560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.593+0000 7f4969ffb640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49840b4830 0x7f4984201020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.593+0000 7f4969ffb640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f497006f500 0x7f4970071a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.593+0000 7f4969ffb640 1 --2- 172.21.15.106:0/2237175706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49840b1f90 0x7f49841fe5e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.593+0000 7f4969ffb640 1 -- 172.21.15.106:0/2237175706 >> 172.21.15.106:0/2237175706 conn(0x7f498409ba30 msgr2=0x7f4984070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.593+0000 7f4969ffb640 1 -- 172.21.15.106:0/2237175706 shutdown_connections 2024-04-25T11:19:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.593+0000 7f4969ffb640 1 -- 172.21.15.106:0/2237175706 wait complete. 2024-04-25T11:19:35.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-04-25T11:19:35.594 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:35.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.613+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f90d0005860 con 0x7f90e80dd850 2024-04-25T11:19:35.612 INFO:teuthology.orchestra.run.smithi106.stdout:55834574857 2024-04-25T11:19:35.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.613+0000 7f90ecff9640 1 -- 172.21.15.106:0/2327245632 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90f0171e10 con 0x7f90e80dd850 2024-04-25T11:19:35.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f90d406f5a0 msgr2=0x7f90d4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f90d406f5a0 0x7f90d4071ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90dc001060 tx=0x7f90dc0079d0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80dd850 msgr2=0x7f90e80d8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80dd850 0x7f90e80d8960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90f0065bf0 tx=0x7f90f0096530 comp rx=0 tx=0).stop 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 shutdown_connections 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e80dd850 0x7f90e80d8960 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e80dd440 0x7f90e80d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f90d406f5a0 0x7f90d4071ab0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 --2- 172.21.15.106:0/2327245632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e80cb010 0x7f90e80e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 >> 172.21.15.106:0/2327245632 conn(0x7f90e8054d00 msgr2=0x7f90e80e0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 shutdown_connections 2024-04-25T11:19:35.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7f90f55c0640 1 -- 172.21.15.106:0/2327245632 wait complete. 2024-04-25T11:19:35.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-04-25T11:19:35.617 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:35.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.617+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd4c00059d0 con 0x7fd4c0057620 2024-04-25T11:19:35.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.621+0000 7fd4c4ff9640 1 -- 172.21.15.106:0/1515510506 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4b4063410 con 0x7fd4c0057620 2024-04-25T11:19:35.618 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:35.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.621+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff5b0005860 con 0x7ff5c8072ac0 2024-04-25T11:19:35.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.621+0000 7ff5be7fc640 1 -- 172.21.15.106:0/736954099 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff5b8016070 con 0x7ff5c8072ac0 2024-04-25T11:19:35.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.621+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f54f4206ce0 con 0x7f54f40cafa0 2024-04-25T11:19:35.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4ac06f400 msgr2=0x7fd4ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4ac06f400 0x7fd4ac071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097f0 tx=0x7fd4bc0a6000 comp rx=0 tx=0).stop 2024-04-25T11:19:35.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 msgr2=0x7fd4c00d8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c00d8e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c4b0 tx=0x7fd4b400c970 comp rx=0 tx=0).stop 2024-04-25T11:19:35.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 shutdown_connections 2024-04-25T11:19:35.623 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:35.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4ac06f400 0x7fd4ac071910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c00582e0 0x7fd4c00d93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4c0057620 0x7fd4c00d8e70 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c4b0 tx=0x7fd4b400c970 comp rx=0 tx=0).stop 2024-04-25T11:19:35.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 --2- 172.21.15.106:0/1515510506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4c003e430 0x7fd4c00d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 >> 172.21.15.106:0/1515510506 conn(0x7fd4c003ac60 msgr2=0x7fd4c00e4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 shutdown_connections 2024-04-25T11:19:35.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7fd4cd0c3640 1 -- 172.21.15.106:0/1515510506 wait complete. 2024-04-25T11:19:35.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff5ac06f430 msgr2=0x7ff5ac071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff5ac06f430 0x7ff5ac071940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8152910 tx=0x7ff5c4009060 comp rx=0 tx=0).stop 2024-04-25T11:19:35.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5c8072ac0 msgr2=0x7ff5c8143f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5c8072ac0 0x7ff5c8143f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8019090 tx=0x7ff5b8019220 comp rx=0 tx=0).stop 2024-04-25T11:19:35.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 shutdown_connections 2024-04-25T11:19:35.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c815ce30 0x7ff5c815e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c80b6030 0x7ff5c8144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff5ac06f430 0x7ff5ac071940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7f54ea7fc640 1 -- 172.21.15.106:0/2054618215 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f54f005ec30 con 0x7f54f40cafa0 2024-04-25T11:19:35.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 --2- 172.21.15.106:0/736954099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5c8072ac0 0x7ff5c8143f30 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8019090 tx=0x7ff5b8019220 comp rx=0 tx=0).stop 2024-04-25T11:19:35.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 >> 172.21.15.106:0/736954099 conn(0x7ff5c80d13f0 msgr2=0x7ff5c80b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 shutdown_connections 2024-04-25T11:19:35.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2260005860 con 0x7f22780b2de0 2024-04-25T11:19:35.627 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T11:19:35.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.625+0000 7f226e7fc640 1 -- 172.21.15.106:0/1397931450 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f227400d040 con 0x7f22780b2de0 2024-04-25T11:19:35.627 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7ff5d0143640 1 -- 172.21.15.106:0/736954099 wait complete. 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f54d806f400 msgr2=0x7f54d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f54d806f400 0x7f54d8071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00a0d0 tx=0x7f54ec0af040 comp rx=0 tx=0).stop 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f40cafa0 msgr2=0x7f54f414c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f40cafa0 0x7f54f414c950 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f54f40d3570 tx=0x7f54f000e830 comp rx=0 tx=0).stop 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 shutdown_connections 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f54d806f400 0x7f54d8071910 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f40cafa0 0x7f54f414c950 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f40cab90 0x7f54f414c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 --2- 172.21.15.106:0/2054618215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00039c0 0x7f54f420a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 >> 172.21.15.106:0/2054618215 conn(0x7f54f40b4020 msgr2=0x7f54f40b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 shutdown_connections 2024-04-25T11:19:35.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f54fbc5b640 1 -- 172.21.15.106:0/2054618215 wait complete. 2024-04-25T11:19:35.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.629+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0600005860 con 0x7f06180d23b0 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f226406f590 msgr2=0x7f2264071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f226406f590 0x7f2264071aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22680ad000 tx=0x7f22680a6000 comp rx=0 tx=0).stop 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22780b2de0 msgr2=0x7f227814cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22780b2de0 0x7f227814cd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f227400b440 tx=0x7f227400b900 comp rx=0 tx=0).stop 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 shutdown_connections 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f226406f590 0x7f2264071aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2278160750 0x7f22780732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2278160340 0x7f22780c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 --2- 172.21.15.106:0/1397931450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22780b2de0 0x7f227814cd50 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f227400b440 tx=0x7f227400b900 comp rx=0 tx=0).stop 2024-04-25T11:19:35.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 >> 172.21.15.106:0/1397931450 conn(0x7f227806cbf0 msgr2=0x7f2278162a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 shutdown_connections 2024-04-25T11:19:35.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f227fafb640 1 -- 172.21.15.106:0/1397931450 wait complete. 2024-04-25T11:19:35.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.633+0000 7f060effd640 1 -- 172.21.15.106:0/3009980143 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f061809fe60 con 0x7f06180d23b0 2024-04-25T11:19:35.633 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T11:19:35.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05fc06f400 msgr2=0x7f05fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05fc06f400 0x7f05fc071910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f06100ae000 tx=0x7f06100a7070 comp rx=0 tx=0).stop 2024-04-25T11:19:35.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06180d23b0 msgr2=0x7f06181fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06180d23b0 0x7f06181fb8a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f061400c3c0 tx=0x7f061400c880 comp rx=0 tx=0).stop 2024-04-25T11:19:35.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 shutdown_connections 2024-04-25T11:19:35.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05fc06f400 0x7f05fc071910 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06180d23b0 0x7f06181fb8a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06180d1fa0 0x7f0618205f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 --2- 172.21.15.106:0/3009980143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06180d17b0 0x7f061809d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 >> 172.21.15.106:0/3009980143 conn(0x7f06180cfe90 msgr2=0x7f06180b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 shutdown_connections 2024-04-25T11:19:35.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.637+0000 7f062031a640 1 -- 172.21.15.106:0/3009980143 wait complete. 2024-04-25T11:19:35.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2024-04-25T11:19:35.648 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:35.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-25T11:19:35.656 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:35.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-25T11:19:35.664 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:35.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-25T11:19:35.665 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:35.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-25T11:19:35.669 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:35.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-25T11:19:35.670 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:35.670 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:19:35.671 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:19:35.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411fabb640 1 Processor -- start 2024-04-25T11:19:35.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411fabb640 1 -- start start 2024-04-25T11:19:35.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411fabb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4118073460 0x7f41180d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411fabb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4118073870 con 0x7f41180cef60 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411fabb640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41180ce050 con 0x7f411815e0a0 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411fabb640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f411815e6b0 con 0x7f4118073460 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411d056640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f41180cef60 0x7f41180d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54246/0 (socket says 172.21.15.106:54246) 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411d056640 1 -- 172.21.15.106:0/1395193280 learned_addr learned my addr 172.21.15.106:0/1395193280 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411e058640 1 --2- 172.21.15.106:0/1395193280 >> v2:172.21.15.134:3300/0 conn(0x7f4118073460 0x7f41180d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525400054 0 0) 0x7f4118073870 con 0x7f41180cef60 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4104003530 con 0x7f41180cef60 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1334728780 0 0) 0x7f4104003530 con 0x7f41180cef60 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 >> v2:172.21.15.134:3300/0 conn(0x7f4118073460 msgr2=0x7f41180d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 --2- 172.21.15.106:0/1395193280 >> v2:172.21.15.134:3300/0 conn(0x7f4118073460 0x7f41180d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 >> v1:172.21.15.134:6789/0 conn(0x7f411815e0a0 legacy=0x7f41180ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411815e990 con 0x7f41180cef60 2024-04-25T11:19:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f4108001680 con 0x7f41180cef60 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 >> v1:172.21.15.106:6789/0 conn(0x7f41180cef60 legacy=0x7f41180d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040039c0 0x7f4104003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4104004810 0x7f4104004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41040051d0 0x7f410400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f411815e6b0 con 0x7f41040051d0 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f410400c170 con 0x7f41040039c0 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f410400c370 con 0x7f4104004810 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411e058640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41040051d0 0x7f410400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411d857640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040039c0 0x7f4104003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411e058640 1 -- 172.21.15.106:0/1395193280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4104004810 msgr2=0x7f4104004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411e058640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4104004810 0x7f4104004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411e058640 1 -- 172.21.15.106:0/1395193280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040039c0 msgr2=0x7f4104003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411e058640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040039c0 0x7f4104003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.845+0000 7f411e058640 1 -- 172.21.15.106:0/1395193280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4104003530 con 0x7f41040051d0 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411e058640 1 -- 172.21.15.106:0/1395193280 reap_dead start 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411e058640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41040051d0 0x7f410400b9d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4110009fc0 tx=0x7f411001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2667837712329017 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110009c10 con 0x7f41040051d0 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41100a5c50 con 0x7f41040051d0 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411c855640 1 -- 172.21.15.106:0/1395193280 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41100039a0 con 0x7f41040051d0 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411fabb640 1 -- 172.21.15.106:0/1395193280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41040051d0 msgr2=0x7f410400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411fabb640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41040051d0 0x7f410400b9d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4110009fc0 tx=0x7f411001cc00 comp rx=0 tx=0).stop 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411fabb640 1 -- 172.21.15.106:0/1395193280 shutdown_connections 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411fabb640 1 --2- 172.21.15.106:0/1395193280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41040051d0 0x7f410400b9d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411fabb640 1 -- 172.21.15.106:0/1395193280 >> 172.21.15.106:0/1395193280 conn(0x7f411806c7c0 msgr2=0x7f411806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411fabb640 1 -- 172.21.15.106:0/1395193280 shutdown_connections 2024-04-25T11:19:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.849+0000 7f411fabb640 1 -- 172.21.15.106:0/1395193280 wait complete. 2024-04-25T11:19:35.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 Processor -- start 2024-04-25T11:19:35.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 -- start start 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040051d0 0x7f41181f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41181f8ab0 0x7f41181453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4118145920 0x7f4118153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f411815ef50 con 0x7f41181f8ab0 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f411815e4b0 con 0x7f41040051d0 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f41180d06e0 con 0x7f4118145920 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41181f8ab0 0x7f41181453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41181f8ab0 0x7f41181453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44478/0 (socket says 172.21.15.106:44478) 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 -- 172.21.15.106:0/3418642027 learned_addr learned my addr 172.21.15.106:0/3418642027 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 -- 172.21.15.106:0/3418642027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4118145920 msgr2=0x7f4118153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4118145920 0x7f4118153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d857640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040051d0 0x7f41181f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 -- 172.21.15.106:0/3418642027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040051d0 msgr2=0x7f41181f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040051d0 0x7f41181f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 -- 172.21.15.106:0/3418642027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4118144620 con 0x7f41181f8ab0 2024-04-25T11:19:35.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411d056640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41181f8ab0 0x7f41181453e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f410800c730 tx=0x7f410800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f410e7fc640 1 -- 172.21.15.106:0/3418642027 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108018070 con 0x7f41181f8ab0 2024-04-25T11:19:35.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f410e7fc640 1 -- 172.21.15.106:0/3418642027 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41080074e0 con 0x7f41181f8ab0 2024-04-25T11:19:35.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4118144900 con 0x7f41181f8ab0 2024-04-25T11:19:35.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f410e7fc640 1 -- 172.21.15.106:0/3418642027 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108004a30 con 0x7f41181f8ab0 2024-04-25T11:19:35.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411815d690 con 0x7f41181f8ab0 2024-04-25T11:19:35.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.853+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4118143360 con 0x7f41181f8ab0 2024-04-25T11:19:35.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.857+0000 7f410e7fc640 1 -- 172.21.15.106:0/3418642027 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f410801b070 con 0x7f41181f8ab0 2024-04-25T11:19:35.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.857+0000 7f410e7fc640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40fc06f430 0x7f40fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.857+0000 7f411d857640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40fc06f430 0x7f40fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.857+0000 7f410e7fc640 1 -- 172.21.15.106:0/3418642027 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f410808e820 con 0x7f41181f8ab0 2024-04-25T11:19:35.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.857+0000 7f411d857640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40fc06f430 0x7f40fc071940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41140015a0 tx=0x7f4114009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:35.861+0000 7f410e7fc640 1 -- 172.21.15.106:0/3418642027 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f410805b710 con 0x7f41181f8ab0 2024-04-25T11:19:36.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.005+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41180b6350 con 0x7f40fc06f430 2024-04-25T11:19:36.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.013+0000 7f410e7fc640 1 -- 172.21.15.106:0/3418642027 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251885 (secure 0 0 0) 0x7f41180b6350 con 0x7f40fc06f430 2024-04-25T11:19:36.016 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:36.018 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40fc06f430 msgr2=0x7f40fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40fc06f430 0x7f40fc071940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41140015a0 tx=0x7f4114009040 comp rx=0 tx=0).stop 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41181f8ab0 msgr2=0x7f41181453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41181f8ab0 0x7f41181453e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f410800c730 tx=0x7f410800cbf0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 shutdown_connections 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4118145920 0x7f4118153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40fc06f430 0x7f40fc071940 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41181f8ab0 0x7f41181453e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 --2- 172.21.15.106:0/3418642027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41040051d0 0x7f41181f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 >> 172.21.15.106:0/3418642027 conn(0x7f411806c7c0 msgr2=0x7f411806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 shutdown_connections 2024-04-25T11:19:36.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.021+0000 7f411fabb640 1 -- 172.21.15.106:0/3418642027 wait complete. 2024-04-25T11:19:36.036 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-25T11:19:34.093133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.690917"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804327+0000","last_change":"2024-04-25T11:19:27.448704+0000","last_active":"2024-04-25T11:19:29.804327+0000","last_peered":"2024-04-25T11:19:29.804327+0000","last_clean":"2024-04-25T11:19:29.804327+0000","last_became_active":"2024-04-25T11:19:27.447782+0000","last_became_peered":"2024-04-25T11:19:27.447782+0000","last_unstale":"2024-04-25T11:19:29.804327+0000","last_undegraded":"2024-04-25T11:19:29.804327+0000","last_fullsized":"2024-04-25T11:19:29.804327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.297417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422162+0000","last_change":"2024-04-25T11:19:27.424499+0000","last_active":"2024-04-25T11:19:28.422162+0000","last_peered":"2024-04-25T11:19:28.422162+0000","last_clean":"2024-04-25T11:19:28.422162+0000","last_became_active":"2024-04-25T11:19:27.424122+0000","last_became_peered":"2024-04-25T11:19:27.424122+0000","last_unstale":"2024-04-25T11:19:28.422162+0000","last_undegraded":"2024-04-25T11:19:28.422162+0000","last_fullsized":"2024-04-25T11:19:28.422162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800964+0000","last_change":"2024-04-25T11:19:27.422128+0000","last_active":"2024-04-25T11:19:29.800964+0000","last_peered":"2024-04-25T11:19:29.800964+0000","last_clean":"2024-04-25T11:19:29.800964+0000","last_became_active":"2024-04-25T11:19:27.421683+0000","last_became_peered":"2024-04-25T11:19:27.421683+0000","last_unstale":"2024-04-25T11:19:29.800964+0000","last_undegraded":"2024-04-25T11:19:29.800964+0000","last_fullsized":"2024-04-25T11:19:29.800964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818544+0000","last_change":"2024-04-25T11:19:27.439794+0000","last_active":"2024-04-25T11:19:28.818544+0000","last_peered":"2024-04-25T11:19:28.818544+0000","last_clean":"2024-04-25T11:19:28.818544+0000","last_became_active":"2024-04-25T11:19:27.439092+0000","last_became_peered":"2024-04-25T11:19:27.439092+0000","last_unstale":"2024-04-25T11:19:28.818544+0000","last_undegraded":"2024-04-25T11:19:28.818544+0000","last_fullsized":"2024-04-25T11:19:28.818544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:52.757499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803029+0000","last_change":"2024-04-25T11:19:27.447315+0000","last_active":"2024-04-25T11:19:29.803029+0000","last_peered":"2024-04-25T11:19:29.803029+0000","last_clean":"2024-04-25T11:19:29.803029+0000","last_became_active":"2024-04-25T11:19:27.446274+0000","last_became_peered":"2024-04-25T11:19:27.446274+0000","last_unstale":"2024-04-25T11:19:29.803029+0000","last_undegraded":"2024-04-25T11:19:29.803029+0000","last_fullsized":"2024-04-25T11:19:29.803029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:01.196607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817544+0000","last_change":"2024-04-25T11:19:27.446758+0000","last_active":"2024-04-25T11:19:28.817544+0000","last_peered":"2024-04-25T11:19:28.817544+0000","last_clean":"2024-04-25T11:19:28.817544+0000","last_became_active":"2024-04-25T11:19:27.440252+0000","last_became_peered":"2024-04-25T11:19:27.440252+0000","last_unstale":"2024-04-25T11:19:28.817544+0000","last_undegraded":"2024-04-25T11:19:28.817544+0000","last_fullsized":"2024-04-25T11:19:28.817544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:08.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803004+0000","last_change":"2024-04-25T11:19:27.442783+0000","last_active":"2024-04-25T11:19:29.803004+0000","last_peered":"2024-04-25T11:19:29.803004+0000","last_clean":"2024-04-25T11:19:29.803004+0000","last_became_active":"2024-04-25T11:19:27.440913+0000","last_became_peered":"2024-04-25T11:19:27.440913+0000","last_unstale":"2024-04-25T11:19:29.803004+0000","last_undegraded":"2024-04-25T11:19:29.803004+0000","last_fullsized":"2024-04-25T11:19:29.803004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.329120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791320+0000","last_change":"2024-04-25T11:19:27.441574+0000","last_active":"2024-04-25T11:19:29.791320+0000","last_peered":"2024-04-25T11:19:29.791320+0000","last_clean":"2024-04-25T11:19:29.791320+0000","last_became_active":"2024-04-25T11:19:27.441265+0000","last_became_peered":"2024-04-25T11:19:27.441265+0000","last_unstale":"2024-04-25T11:19:29.791320+0000","last_undegraded":"2024-04-25T11:19:29.791320+0000","last_fullsized":"2024-04-25T11:19:29.791320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804555+0000","last_change":"2024-04-25T11:19:27.459170+0000","last_active":"2024-04-25T11:19:29.804555+0000","last_peered":"2024-04-25T11:19:29.804555+0000","last_clean":"2024-04-25T11:19:29.804555+0000","last_became_active":"2024-04-25T11:19:27.458755+0000","last_became_peered":"2024-04-25T11:19:27.458755+0000","last_unstale":"2024-04-25T11:19:29.804555+0000","last_undegraded":"2024-04-25T11:19:29.804555+0000","last_fullsized":"2024-04-25T11:19:29.804555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:52.797941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433030+0000","last_change":"2024-04-25T11:19:27.456078+0000","last_active":"2024-04-25T11:19:28.433030+0000","last_peered":"2024-04-25T11:19:28.433030+0000","last_clean":"2024-04-25T11:19:28.433030+0000","last_became_active":"2024-04-25T11:19:27.455302+0000","last_became_peered":"2024-04-25T11:19:27.455302+0000","last_unstale":"2024-04-25T11:19:28.433030+0000","last_undegraded":"2024-04-25T11:19:28.433030+0000","last_fullsized":"2024-04-25T11:19:28.433030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432803+0000","last_change":"2024-04-25T11:19:27.459132+0000","last_active":"2024-04-25T11:19:28.432803+0000","last_peered":"2024-04-25T11:19:28.432803+0000","last_clean":"2024-04-25T11:19:28.432803+0000","last_became_active":"2024-04-25T11:19:27.458820+0000","last_became_peered":"2024-04-25T11:19:27.458820+0000","last_unstale":"2024-04-25T11:19:28.432803+0000","last_undegraded":"2024-04-25T11:19:28.432803+0000","last_fullsized":"2024-04-25T11:19:28.432803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.067341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802226+0000","last_change":"2024-04-25T11:19:27.439470+0000","last_active":"2024-04-25T11:19:29.802226+0000","last_peered":"2024-04-25T11:19:29.802226+0000","last_clean":"2024-04-25T11:19:29.802226+0000","last_became_active":"2024-04-25T11:19:27.439141+0000","last_became_peered":"2024-04-25T11:19:27.439141+0000","last_unstale":"2024-04-25T11:19:29.802226+0000","last_undegraded":"2024-04-25T11:19:29.802226+0000","last_fullsized":"2024-04-25T11:19:29.802226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.059005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791949+0000","last_change":"2024-04-25T11:19:27.433821+0000","last_active":"2024-04-25T11:19:29.791949+0000","last_peered":"2024-04-25T11:19:29.791949+0000","last_clean":"2024-04-25T11:19:29.791949+0000","last_became_active":"2024-04-25T11:19:27.433427+0000","last_became_peered":"2024-04-25T11:19:27.433427+0000","last_unstale":"2024-04-25T11:19:29.791949+0000","last_undegraded":"2024-04-25T11:19:29.791949+0000","last_fullsized":"2024-04-25T11:19:29.791949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800437+0000","last_change":"2024-04-25T11:19:27.428346+0000","last_active":"2024-04-25T11:19:29.800437+0000","last_peered":"2024-04-25T11:19:29.800437+0000","last_clean":"2024-04-25T11:19:29.800437+0000","last_became_active":"2024-04-25T11:19:27.428006+0000","last_became_peered":"2024-04-25T11:19:27.428006+0000","last_unstale":"2024-04-25T11:19:29.800437+0000","last_undegraded":"2024-04-25T11:19:29.800437+0000","last_fullsized":"2024-04-25T11:19:29.800437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:41.086001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421393+0000","last_change":"2024-04-25T11:19:27.428566+0000","last_active":"2024-04-25T11:19:28.421393+0000","last_peered":"2024-04-25T11:19:28.421393+0000","last_clean":"2024-04-25T11:19:28.421393+0000","last_became_active":"2024-04-25T11:19:27.427282+0000","last_became_peered":"2024-04-25T11:19:27.427282+0000","last_unstale":"2024-04-25T11:19:28.421393+0000","last_undegraded":"2024-04-25T11:19:28.421393+0000","last_fullsized":"2024-04-25T11:19:28.421393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817322+0000","last_change":"2024-04-25T11:19:27.448380+0000","last_active":"2024-04-25T11:19:28.817322+0000","last_peered":"2024-04-25T11:19:28.817322+0000","last_clean":"2024-04-25T11:19:28.817322+0000","last_became_active":"2024-04-25T11:19:27.447971+0000","last_became_peered":"2024-04-25T11:19:27.447971+0000","last_unstale":"2024-04-25T11:19:28.817322+0000","last_undegraded":"2024-04-25T11:19:28.817322+0000","last_fullsized":"2024-04-25T11:19:28.817322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.850268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818713+0000","last_change":"2024-04-25T11:19:27.452389+0000","last_active":"2024-04-25T11:19:28.818713+0000","last_peered":"2024-04-25T11:19:28.818713+0000","last_clean":"2024-04-25T11:19:28.818713+0000","last_became_active":"2024-04-25T11:19:27.451910+0000","last_became_peered":"2024-04-25T11:19:27.451910+0000","last_unstale":"2024-04-25T11:19:28.818713+0000","last_undegraded":"2024-04-25T11:19:28.818713+0000","last_fullsized":"2024-04-25T11:19:28.818713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424567+0000","last_change":"2024-04-25T11:19:27.437476+0000","last_active":"2024-04-25T11:19:28.424567+0000","last_peered":"2024-04-25T11:19:28.424567+0000","last_clean":"2024-04-25T11:19:28.424567+0000","last_became_active":"2024-04-25T11:19:27.436712+0000","last_became_peered":"2024-04-25T11:19:27.436712+0000","last_unstale":"2024-04-25T11:19:28.424567+0000","last_undegraded":"2024-04-25T11:19:28.424567+0000","last_fullsized":"2024-04-25T11:19:28.424567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804153+0000","last_change":"2024-04-25T11:19:27.445559+0000","last_active":"2024-04-25T11:19:29.804153+0000","last_peered":"2024-04-25T11:19:29.804153+0000","last_clean":"2024-04-25T11:19:29.804153+0000","last_became_active":"2024-04-25T11:19:27.445058+0000","last_became_peered":"2024-04-25T11:19:27.445058+0000","last_unstale":"2024-04-25T11:19:29.804153+0000","last_undegraded":"2024-04-25T11:19:29.804153+0000","last_fullsized":"2024-04-25T11:19:29.804153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432846+0000","last_change":"2024-04-25T11:19:27.441427+0000","last_active":"2024-04-25T11:19:28.432846+0000","last_peered":"2024-04-25T11:19:28.432846+0000","last_clean":"2024-04-25T11:19:28.432846+0000","last_became_active":"2024-04-25T11:19:27.441087+0000","last_became_peered":"2024-04-25T11:19:27.441087+0000","last_unstale":"2024-04-25T11:19:28.432846+0000","last_undegraded":"2024-04-25T11:19:28.432846+0000","last_fullsized":"2024-04-25T11:19:28.432846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801155+0000","last_change":"2024-04-25T11:19:27.445781+0000","last_active":"2024-04-25T11:19:29.801155+0000","last_peered":"2024-04-25T11:19:29.801155+0000","last_clean":"2024-04-25T11:19:29.801155+0000","last_became_active":"2024-04-25T11:19:27.445240+0000","last_became_peered":"2024-04-25T11:19:27.445240+0000","last_unstale":"2024-04-25T11:19:29.801155+0000","last_undegraded":"2024-04-25T11:19:29.801155+0000","last_fullsized":"2024-04-25T11:19:29.801155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817748+0000","last_change":"2024-04-25T11:19:27.453395+0000","last_active":"2024-04-25T11:19:28.817748+0000","last_peered":"2024-04-25T11:19:28.817748+0000","last_clean":"2024-04-25T11:19:28.817748+0000","last_became_active":"2024-04-25T11:19:27.452942+0000","last_became_peered":"2024-04-25T11:19:27.452942+0000","last_unstale":"2024-04-25T11:19:28.817748+0000","last_undegraded":"2024-04-25T11:19:28.817748+0000","last_fullsized":"2024-04-25T11:19:28.817748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:39.163529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791445+0000","last_change":"2024-04-25T11:19:27.427559+0000","last_active":"2024-04-25T11:19:29.791445+0000","last_peered":"2024-04-25T11:19:29.791445+0000","last_clean":"2024-04-25T11:19:29.791445+0000","last_became_active":"2024-04-25T11:19:27.421099+0000","last_became_peered":"2024-04-25T11:19:27.421099+0000","last_unstale":"2024-04-25T11:19:29.791445+0000","last_undegraded":"2024-04-25T11:19:29.791445+0000","last_fullsized":"2024-04-25T11:19:29.791445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817459+0000","last_change":"2024-04-25T11:19:27.440162+0000","last_active":"2024-04-25T11:19:28.817459+0000","last_peered":"2024-04-25T11:19:28.817459+0000","last_clean":"2024-04-25T11:19:28.817459+0000","last_became_active":"2024-04-25T11:19:27.439384+0000","last_became_peered":"2024-04-25T11:19:27.439384+0000","last_unstale":"2024-04-25T11:19:28.817459+0000","last_undegraded":"2024-04-25T11:19:28.817459+0000","last_fullsized":"2024-04-25T11:19:28.817459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.398422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792627+0000","last_change":"2024-04-25T11:19:27.438911+0000","last_active":"2024-04-25T11:19:29.792627+0000","last_peered":"2024-04-25T11:19:29.792627+0000","last_clean":"2024-04-25T11:19:29.792627+0000","last_became_active":"2024-04-25T11:19:27.437950+0000","last_became_peered":"2024-04-25T11:19:27.437950+0000","last_unstale":"2024-04-25T11:19:29.792627+0000","last_undegraded":"2024-04-25T11:19:29.792627+0000","last_fullsized":"2024-04-25T11:19:29.792627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:08.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433178+0000","last_change":"2024-04-25T11:19:27.447613+0000","last_active":"2024-04-25T11:19:28.433178+0000","last_peered":"2024-04-25T11:19:28.433178+0000","last_clean":"2024-04-25T11:19:28.433178+0000","last_became_active":"2024-04-25T11:19:27.447314+0000","last_became_peered":"2024-04-25T11:19:27.447314+0000","last_unstale":"2024-04-25T11:19:28.433178+0000","last_undegraded":"2024-04-25T11:19:28.433178+0000","last_fullsized":"2024-04-25T11:19:28.433178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421840+0000","last_change":"2024-04-25T11:19:27.436767+0000","last_active":"2024-04-25T11:19:28.421840+0000","last_peered":"2024-04-25T11:19:28.421840+0000","last_clean":"2024-04-25T11:19:28.421840+0000","last_became_active":"2024-04-25T11:19:27.436392+0000","last_became_peered":"2024-04-25T11:19:27.436392+0000","last_unstale":"2024-04-25T11:19:28.421840+0000","last_undegraded":"2024-04-25T11:19:28.421840+0000","last_fullsized":"2024-04-25T11:19:28.421840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792127+0000","last_change":"2024-04-25T11:19:27.438536+0000","last_active":"2024-04-25T11:19:29.792127+0000","last_peered":"2024-04-25T11:19:29.792127+0000","last_clean":"2024-04-25T11:19:29.792127+0000","last_became_active":"2024-04-25T11:19:27.438198+0000","last_became_peered":"2024-04-25T11:19:27.438198+0000","last_unstale":"2024-04-25T11:19:29.792127+0000","last_undegraded":"2024-04-25T11:19:29.792127+0000","last_fullsized":"2024-04-25T11:19:29.792127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.942052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801440+0000","last_change":"2024-04-25T11:19:27.425325+0000","last_active":"2024-04-25T11:19:29.801440+0000","last_peered":"2024-04-25T11:19:29.801440+0000","last_clean":"2024-04-25T11:19:29.801440+0000","last_became_active":"2024-04-25T11:19:27.423815+0000","last_became_peered":"2024-04-25T11:19:27.423815+0000","last_unstale":"2024-04-25T11:19:29.801440+0000","last_undegraded":"2024-04-25T11:19:29.801440+0000","last_fullsized":"2024-04-25T11:19:29.801440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.942052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818101+0000","last_change":"2024-04-25T11:19:27.447151+0000","last_active":"2024-04-25T11:19:28.818101+0000","last_peered":"2024-04-25T11:19:28.818101+0000","last_clean":"2024-04-25T11:19:28.818101+0000","last_became_active":"2024-04-25T11:19:27.446193+0000","last_became_peered":"2024-04-25T11:19:27.446193+0000","last_unstale":"2024-04-25T11:19:28.818101+0000","last_undegraded":"2024-04-25T11:19:28.818101+0000","last_fullsized":"2024-04-25T11:19:28.818101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:01.196607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424335+0000","last_change":"2024-04-25T11:19:27.440245+0000","last_active":"2024-04-25T11:19:28.424335+0000","last_peered":"2024-04-25T11:19:28.424335+0000","last_clean":"2024-04-25T11:19:28.424335+0000","last_became_active":"2024-04-25T11:19:27.439755+0000","last_became_peered":"2024-04-25T11:19:27.439755+0000","last_unstale":"2024-04-25T11:19:28.424335+0000","last_undegraded":"2024-04-25T11:19:28.424335+0000","last_fullsized":"2024-04-25T11:19:28.424335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433384+0000","last_change":"2024-04-25T11:19:27.433866+0000","last_active":"2024-04-25T11:19:28.433384+0000","last_peered":"2024-04-25T11:19:28.433384+0000","last_clean":"2024-04-25T11:19:28.433384+0000","last_became_active":"2024-04-25T11:19:27.423031+0000","last_became_peered":"2024-04-25T11:19:27.423031+0000","last_unstale":"2024-04-25T11:19:28.433384+0000","last_undegraded":"2024-04-25T11:19:28.433384+0000","last_fullsized":"2024-04-25T11:19:28.433384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801476+0000","last_change":"2024-04-25T11:19:27.427233+0000","last_active":"2024-04-25T11:19:29.801476+0000","last_peered":"2024-04-25T11:19:29.801476+0000","last_clean":"2024-04-25T11:19:29.801476+0000","last_became_active":"2024-04-25T11:19:27.426438+0000","last_became_peered":"2024-04-25T11:19:27.426438+0000","last_unstale":"2024-04-25T11:19:29.801476+0000","last_undegraded":"2024-04-25T11:19:29.801476+0000","last_fullsized":"2024-04-25T11:19:29.801476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:25.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792171+0000","last_change":"2024-04-25T11:19:26.448872+0000","last_active":"2024-04-25T11:19:29.792171+0000","last_peered":"2024-04-25T11:19:29.792171+0000","last_clean":"2024-04-25T11:19:29.792171+0000","last_became_active":"2024-04-25T11:19:26.448492+0000","last_became_peered":"2024-04-25T11:19:26.448492+0000","last_unstale":"2024-04-25T11:19:29.792171+0000","last_undegraded":"2024-04-25T11:19:29.792171+0000","last_fullsized":"2024-04-25T11:19:29.792171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801354+0000","last_change":"2024-04-25T11:19:27.424636+0000","last_active":"2024-04-25T11:19:29.801354+0000","last_peered":"2024-04-25T11:19:29.801354+0000","last_clean":"2024-04-25T11:19:29.801354+0000","last_became_active":"2024-04-25T11:19:27.422364+0000","last_became_peered":"2024-04-25T11:19:27.422364+0000","last_unstale":"2024-04-25T11:19:29.801354+0000","last_undegraded":"2024-04-25T11:19:29.801354+0000","last_fullsized":"2024-04-25T11:19:29.801354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432509+0000","last_change":"2024-04-25T11:19:26.446971+0000","last_active":"2024-04-25T11:19:28.432509+0000","last_peered":"2024-04-25T11:19:28.432509+0000","last_clean":"2024-04-25T11:19:28.432509+0000","last_became_active":"2024-04-25T11:19:26.445751+0000","last_became_peered":"2024-04-25T11:19:26.445751+0000","last_unstale":"2024-04-25T11:19:28.432509+0000","last_undegraded":"2024-04-25T11:19:28.432509+0000","last_fullsized":"2024-04-25T11:19:28.432509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791338+0000","last_change":"2024-04-25T11:19:26.430729+0000","last_active":"2024-04-25T11:19:29.791338+0000","last_peered":"2024-04-25T11:19:29.791338+0000","last_clean":"2024-04-25T11:19:29.791338+0000","last_became_active":"2024-04-25T11:19:26.430240+0000","last_became_peered":"2024-04-25T11:19:26.430240+0000","last_unstale":"2024-04-25T11:19:29.791338+0000","last_undegraded":"2024-04-25T11:19:29.791338+0000","last_fullsized":"2024-04-25T11:19:29.791338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:10.270275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791441+0000","last_change":"2024-04-25T11:19:27.425985+0000","last_active":"2024-04-25T11:19:29.791441+0000","last_peered":"2024-04-25T11:19:29.791441+0000","last_clean":"2024-04-25T11:19:29.791441+0000","last_became_active":"2024-04-25T11:19:27.424091+0000","last_became_peered":"2024-04-25T11:19:27.424091+0000","last_unstale":"2024-04-25T11:19:29.791441+0000","last_undegraded":"2024-04-25T11:19:29.791441+0000","last_fullsized":"2024-04-25T11:19:29.791441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817641+0000","last_change":"2024-04-25T11:19:26.446399+0000","last_active":"2024-04-25T11:19:28.817641+0000","last_peered":"2024-04-25T11:19:28.817641+0000","last_clean":"2024-04-25T11:19:28.817641+0000","last_became_active":"2024-04-25T11:19:26.445502+0000","last_became_peered":"2024-04-25T11:19:26.445502+0000","last_unstale":"2024-04-25T11:19:28.817641+0000","last_undegraded":"2024-04-25T11:19:28.817641+0000","last_fullsized":"2024-04-25T11:19:28.817641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421333+0000","last_change":"2024-04-25T11:19:27.435289+0000","last_active":"2024-04-25T11:19:28.421333+0000","last_peered":"2024-04-25T11:19:28.421333+0000","last_clean":"2024-04-25T11:19:28.421333+0000","last_became_active":"2024-04-25T11:19:27.434888+0000","last_became_peered":"2024-04-25T11:19:27.434888+0000","last_unstale":"2024-04-25T11:19:28.421333+0000","last_undegraded":"2024-04-25T11:19:28.421333+0000","last_fullsized":"2024-04-25T11:19:28.421333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432966+0000","last_change":"2024-04-25T11:19:26.447470+0000","last_active":"2024-04-25T11:19:28.432966+0000","last_peered":"2024-04-25T11:19:28.432966+0000","last_clean":"2024-04-25T11:19:28.432966+0000","last_became_active":"2024-04-25T11:19:26.445469+0000","last_became_peered":"2024-04-25T11:19:26.445469+0000","last_unstale":"2024-04-25T11:19:28.432966+0000","last_undegraded":"2024-04-25T11:19:28.432966+0000","last_fullsized":"2024-04-25T11:19:28.432966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818172+0000","last_change":"2024-04-25T11:19:27.454986+0000","last_active":"2024-04-25T11:19:28.818172+0000","last_peered":"2024-04-25T11:19:28.818172+0000","last_clean":"2024-04-25T11:19:28.818172+0000","last_became_active":"2024-04-25T11:19:27.448543+0000","last_became_peered":"2024-04-25T11:19:27.448543+0000","last_unstale":"2024-04-25T11:19:28.818172+0000","last_undegraded":"2024-04-25T11:19:28.818172+0000","last_fullsized":"2024-04-25T11:19:28.818172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.329120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818061+0000","last_change":"2024-04-25T11:19:26.446709+0000","last_active":"2024-04-25T11:19:28.818061+0000","last_peered":"2024-04-25T11:19:28.818061+0000","last_clean":"2024-04-25T11:19:28.818061+0000","last_became_active":"2024-04-25T11:19:26.445742+0000","last_became_peered":"2024-04-25T11:19:26.445742+0000","last_unstale":"2024-04-25T11:19:28.818061+0000","last_undegraded":"2024-04-25T11:19:28.818061+0000","last_fullsized":"2024-04-25T11:19:28.818061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800839+0000","last_change":"2024-04-25T11:19:27.431751+0000","last_active":"2024-04-25T11:19:29.800839+0000","last_peered":"2024-04-25T11:19:29.800839+0000","last_clean":"2024-04-25T11:19:29.800839+0000","last_became_active":"2024-04-25T11:19:27.431063+0000","last_became_peered":"2024-04-25T11:19:27.431063+0000","last_unstale":"2024-04-25T11:19:29.800839+0000","last_undegraded":"2024-04-25T11:19:29.800839+0000","last_fullsized":"2024-04-25T11:19:29.800839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.067341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422020+0000","last_change":"2024-04-25T11:19:26.435639+0000","last_active":"2024-04-25T11:19:28.422020+0000","last_peered":"2024-04-25T11:19:28.422020+0000","last_clean":"2024-04-25T11:19:28.422020+0000","last_became_active":"2024-04-25T11:19:26.435051+0000","last_became_peered":"2024-04-25T11:19:26.435051+0000","last_unstale":"2024-04-25T11:19:28.422020+0000","last_undegraded":"2024-04-25T11:19:28.422020+0000","last_fullsized":"2024-04-25T11:19:28.422020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802432+0000","last_change":"2024-04-25T11:19:27.449942+0000","last_active":"2024-04-25T11:19:29.802432+0000","last_peered":"2024-04-25T11:19:29.802432+0000","last_clean":"2024-04-25T11:19:29.802432+0000","last_became_active":"2024-04-25T11:19:27.449419+0000","last_became_peered":"2024-04-25T11:19:27.449419+0000","last_unstale":"2024-04-25T11:19:29.802432+0000","last_undegraded":"2024-04-25T11:19:29.802432+0000","last_fullsized":"2024-04-25T11:19:29.802432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432820+0000","last_change":"2024-04-25T11:19:26.437548+0000","last_active":"2024-04-25T11:19:28.432820+0000","last_peered":"2024-04-25T11:19:28.432820+0000","last_clean":"2024-04-25T11:19:28.432820+0000","last_became_active":"2024-04-25T11:19:26.436659+0000","last_became_peered":"2024-04-25T11:19:26.436659+0000","last_unstale":"2024-04-25T11:19:28.432820+0000","last_undegraded":"2024-04-25T11:19:28.432820+0000","last_fullsized":"2024-04-25T11:19:28.432820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803625+0000","last_change":"2024-04-25T11:19:27.443841+0000","last_active":"2024-04-25T11:19:29.803625+0000","last_peered":"2024-04-25T11:19:29.803625+0000","last_clean":"2024-04-25T11:19:29.803625+0000","last_became_active":"2024-04-25T11:19:27.443174+0000","last_became_peered":"2024-04-25T11:19:27.443174+0000","last_unstale":"2024-04-25T11:19:29.803625+0000","last_undegraded":"2024-04-25T11:19:29.803625+0000","last_fullsized":"2024-04-25T11:19:29.803625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:52.757499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801465+0000","last_change":"2024-04-25T11:19:26.444467+0000","last_active":"2024-04-25T11:19:29.801465+0000","last_peered":"2024-04-25T11:19:29.801465+0000","last_clean":"2024-04-25T11:19:29.801465+0000","last_became_active":"2024-04-25T11:19:26.444081+0000","last_became_peered":"2024-04-25T11:19:26.444081+0000","last_unstale":"2024-04-25T11:19:29.801465+0000","last_undegraded":"2024-04-25T11:19:29.801465+0000","last_fullsized":"2024-04-25T11:19:29.801465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:10.270275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800928+0000","last_change":"2024-04-25T11:19:27.442198+0000","last_active":"2024-04-25T11:19:29.800928+0000","last_peered":"2024-04-25T11:19:29.800928+0000","last_clean":"2024-04-25T11:19:29.800928+0000","last_became_active":"2024-04-25T11:19:27.441771+0000","last_became_peered":"2024-04-25T11:19:27.441771+0000","last_unstale":"2024-04-25T11:19:29.800928+0000","last_undegraded":"2024-04-25T11:19:29.800928+0000","last_fullsized":"2024-04-25T11:19:29.800928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818060+0000","last_change":"2024-04-25T11:19:26.443383+0000","last_active":"2024-04-25T11:19:28.818060+0000","last_peered":"2024-04-25T11:19:28.818060+0000","last_clean":"2024-04-25T11:19:28.818060+0000","last_became_active":"2024-04-25T11:19:26.442678+0000","last_became_peered":"2024-04-25T11:19:26.442678+0000","last_unstale":"2024-04-25T11:19:28.818060+0000","last_undegraded":"2024-04-25T11:19:28.818060+0000","last_fullsized":"2024-04-25T11:19:28.818060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803299+0000","last_change":"2024-04-25T11:19:27.448069+0000","last_active":"2024-04-25T11:19:29.803299+0000","last_peered":"2024-04-25T11:19:29.803299+0000","last_clean":"2024-04-25T11:19:29.803299+0000","last_became_active":"2024-04-25T11:19:27.447313+0000","last_became_peered":"2024-04-25T11:19:27.447313+0000","last_unstale":"2024-04-25T11:19:29.803299+0000","last_undegraded":"2024-04-25T11:19:29.803299+0000","last_fullsized":"2024-04-25T11:19:29.803299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:03.621864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432496+0000","last_change":"2024-04-25T11:19:26.449988+0000","last_active":"2024-04-25T11:19:28.432496+0000","last_peered":"2024-04-25T11:19:28.432496+0000","last_clean":"2024-04-25T11:19:28.432496+0000","last_became_active":"2024-04-25T11:19:26.445253+0000","last_became_peered":"2024-04-25T11:19:26.445253+0000","last_unstale":"2024-04-25T11:19:28.432496+0000","last_undegraded":"2024-04-25T11:19:28.432496+0000","last_fullsized":"2024-04-25T11:19:28.432496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432903+0000","last_change":"2024-04-25T11:19:27.423470+0000","last_active":"2024-04-25T11:19:28.432903+0000","last_peered":"2024-04-25T11:19:28.432903+0000","last_clean":"2024-04-25T11:19:28.432903+0000","last_became_active":"2024-04-25T11:19:27.422649+0000","last_became_peered":"2024-04-25T11:19:27.422649+0000","last_unstale":"2024-04-25T11:19:28.432903+0000","last_undegraded":"2024-04-25T11:19:28.432903+0000","last_fullsized":"2024-04-25T11:19:28.432903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.423793+0000","last_change":"2024-04-25T11:19:26.424599+0000","last_active":"2024-04-25T11:19:28.423793+0000","last_peered":"2024-04-25T11:19:28.423793+0000","last_clean":"2024-04-25T11:19:28.423793+0000","last_became_active":"2024-04-25T11:19:26.424110+0000","last_became_peered":"2024-04-25T11:19:26.424110+0000","last_unstale":"2024-04-25T11:19:28.423793+0000","last_undegraded":"2024-04-25T11:19:28.423793+0000","last_fullsized":"2024-04-25T11:19:28.423793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802558+0000","last_change":"2024-04-25T11:19:27.441846+0000","last_active":"2024-04-25T11:19:29.802558+0000","last_peered":"2024-04-25T11:19:29.802558+0000","last_clean":"2024-04-25T11:19:29.802558+0000","last_became_active":"2024-04-25T11:19:27.441349+0000","last_became_peered":"2024-04-25T11:19:27.441349+0000","last_unstale":"2024-04-25T11:19:29.802558+0000","last_undegraded":"2024-04-25T11:19:29.802558+0000","last_fullsized":"2024-04-25T11:19:29.802558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:39.163529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802696+0000","last_change":"2024-04-25T11:19:26.429643+0000","last_active":"2024-04-25T11:19:29.802696+0000","last_peered":"2024-04-25T11:19:29.802696+0000","last_clean":"2024-04-25T11:19:29.802696+0000","last_became_active":"2024-04-25T11:19:26.428760+0000","last_became_peered":"2024-04-25T11:19:26.428760+0000","last_unstale":"2024-04-25T11:19:29.802696+0000","last_undegraded":"2024-04-25T11:19:29.802696+0000","last_fullsized":"2024-04-25T11:19:29.802696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:36.804999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803977+0000","last_change":"2024-04-25T11:19:27.445422+0000","last_active":"2024-04-25T11:19:29.803977+0000","last_peered":"2024-04-25T11:19:29.803977+0000","last_clean":"2024-04-25T11:19:29.803977+0000","last_became_active":"2024-04-25T11:19:27.444978+0000","last_became_peered":"2024-04-25T11:19:27.444978+0000","last_unstale":"2024-04-25T11:19:29.803977+0000","last_undegraded":"2024-04-25T11:19:29.803977+0000","last_fullsized":"2024-04-25T11:19:29.803977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.518039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424380+0000","last_change":"2024-04-25T11:19:26.415971+0000","last_active":"2024-04-25T11:19:28.424380+0000","last_peered":"2024-04-25T11:19:28.424380+0000","last_clean":"2024-04-25T11:19:28.424380+0000","last_became_active":"2024-04-25T11:19:26.415532+0000","last_became_peered":"2024-04-25T11:19:26.415532+0000","last_unstale":"2024-04-25T11:19:28.424380+0000","last_undegraded":"2024-04-25T11:19:28.424380+0000","last_fullsized":"2024-04-25T11:19:28.424380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804821+0000","last_change":"2024-04-25T11:19:27.445828+0000","last_active":"2024-04-25T11:19:29.804821+0000","last_peered":"2024-04-25T11:19:29.804821+0000","last_clean":"2024-04-25T11:19:29.804821+0000","last_became_active":"2024-04-25T11:19:27.445424+0000","last_became_peered":"2024-04-25T11:19:27.445424+0000","last_unstale":"2024-04-25T11:19:29.804821+0000","last_undegraded":"2024-04-25T11:19:29.804821+0000","last_fullsized":"2024-04-25T11:19:29.804821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:41.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422118+0000","last_change":"2024-04-25T11:19:26.442966+0000","last_active":"2024-04-25T11:19:28.422118+0000","last_peered":"2024-04-25T11:19:28.422118+0000","last_clean":"2024-04-25T11:19:28.422118+0000","last_became_active":"2024-04-25T11:19:26.442531+0000","last_became_peered":"2024-04-25T11:19:26.442531+0000","last_unstale":"2024-04-25T11:19:28.422118+0000","last_undegraded":"2024-04-25T11:19:28.422118+0000","last_fullsized":"2024-04-25T11:19:28.422118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818802+0000","last_change":"2024-04-25T11:19:27.454129+0000","last_active":"2024-04-25T11:19:28.818802+0000","last_peered":"2024-04-25T11:19:28.818802+0000","last_clean":"2024-04-25T11:19:28.818802+0000","last_became_active":"2024-04-25T11:19:27.453260+0000","last_became_peered":"2024-04-25T11:19:27.453260+0000","last_unstale":"2024-04-25T11:19:28.818802+0000","last_undegraded":"2024-04-25T11:19:28.818802+0000","last_fullsized":"2024-04-25T11:19:28.818802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.297417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424074+0000","last_change":"2024-04-25T11:19:26.435919+0000","last_active":"2024-04-25T11:19:28.424074+0000","last_peered":"2024-04-25T11:19:28.424074+0000","last_clean":"2024-04-25T11:19:28.424074+0000","last_became_active":"2024-04-25T11:19:26.435172+0000","last_became_peered":"2024-04-25T11:19:26.435172+0000","last_unstale":"2024-04-25T11:19:28.424074+0000","last_undegraded":"2024-04-25T11:19:28.424074+0000","last_fullsized":"2024-04-25T11:19:28.424074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803927+0000","last_change":"2024-04-25T11:19:27.452401+0000","last_active":"2024-04-25T11:19:29.803927+0000","last_peered":"2024-04-25T11:19:29.803927+0000","last_clean":"2024-04-25T11:19:29.803927+0000","last_became_active":"2024-04-25T11:19:27.452007+0000","last_became_peered":"2024-04-25T11:19:27.452007+0000","last_unstale":"2024-04-25T11:19:29.803927+0000","last_undegraded":"2024-04-25T11:19:29.803927+0000","last_fullsized":"2024-04-25T11:19:29.803927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:45.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802096+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:19:29.802096+0000","last_peered":"2024-04-25T11:19:29.802096+0000","last_clean":"2024-04-25T11:19:29.802096+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:19:29.802096+0000","last_undegraded":"2024-04-25T11:19:29.802096+0000","last_fullsized":"2024-04-25T11:19:29.802096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817551+0000","last_change":"2024-04-25T11:19:26.431881+0000","last_active":"2024-04-25T11:19:28.817551+0000","last_peered":"2024-04-25T11:19:28.817551+0000","last_clean":"2024-04-25T11:19:28.817551+0000","last_became_active":"2024-04-25T11:19:26.431224+0000","last_became_peered":"2024-04-25T11:19:26.431224+0000","last_unstale":"2024-04-25T11:19:28.817551+0000","last_undegraded":"2024-04-25T11:19:28.817551+0000","last_fullsized":"2024-04-25T11:19:28.817551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791934+0000","last_change":"2024-04-25T11:19:27.453756+0000","last_active":"2024-04-25T11:19:29.791934+0000","last_peered":"2024-04-25T11:19:29.791934+0000","last_clean":"2024-04-25T11:19:29.791934+0000","last_became_active":"2024-04-25T11:19:27.453406+0000","last_became_peered":"2024-04-25T11:19:27.453406+0000","last_unstale":"2024-04-25T11:19:29.791934+0000","last_undegraded":"2024-04-25T11:19:29.791934+0000","last_fullsized":"2024-04-25T11:19:29.791934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801787+0000","last_change":"2024-04-25T11:19:26.439003+0000","last_active":"2024-04-25T11:19:29.801787+0000","last_peered":"2024-04-25T11:19:29.801787+0000","last_clean":"2024-04-25T11:19:29.801787+0000","last_became_active":"2024-04-25T11:19:26.437864+0000","last_became_peered":"2024-04-25T11:19:26.437864+0000","last_unstale":"2024-04-25T11:19:29.801787+0000","last_undegraded":"2024-04-25T11:19:29.801787+0000","last_fullsized":"2024-04-25T11:19:29.801787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:03.260366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800291+0000","last_change":"2024-04-25T11:19:27.411133+0000","last_active":"2024-04-25T11:19:29.800291+0000","last_peered":"2024-04-25T11:19:29.800291+0000","last_clean":"2024-04-25T11:19:29.800291+0000","last_became_active":"2024-04-25T11:19:27.410234+0000","last_became_peered":"2024-04-25T11:19:27.410234+0000","last_unstale":"2024-04-25T11:19:29.800291+0000","last_undegraded":"2024-04-25T11:19:29.800291+0000","last_fullsized":"2024-04-25T11:19:29.800291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.618925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424634+0000","last_change":"2024-04-25T11:19:26.413057+0000","last_active":"2024-04-25T11:19:28.424634+0000","last_peered":"2024-04-25T11:19:28.424634+0000","last_clean":"2024-04-25T11:19:28.424634+0000","last_became_active":"2024-04-25T11:19:26.412586+0000","last_became_peered":"2024-04-25T11:19:26.412586+0000","last_unstale":"2024-04-25T11:19:28.424634+0000","last_undegraded":"2024-04-25T11:19:28.424634+0000","last_fullsized":"2024-04-25T11:19:28.424634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421086+0000","last_change":"2024-04-25T11:19:27.423812+0000","last_active":"2024-04-25T11:19:28.421086+0000","last_peered":"2024-04-25T11:19:28.421086+0000","last_clean":"2024-04-25T11:19:28.421086+0000","last_became_active":"2024-04-25T11:19:27.419444+0000","last_became_peered":"2024-04-25T11:19:27.419444+0000","last_unstale":"2024-04-25T11:19:28.421086+0000","last_undegraded":"2024-04-25T11:19:28.421086+0000","last_fullsized":"2024-04-25T11:19:28.421086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818406+0000","last_change":"2024-04-25T11:19:26.433315+0000","last_active":"2024-04-25T11:19:28.818406+0000","last_peered":"2024-04-25T11:19:28.818406+0000","last_clean":"2024-04-25T11:19:28.818406+0000","last_became_active":"2024-04-25T11:19:26.432639+0000","last_became_peered":"2024-04-25T11:19:26.432639+0000","last_unstale":"2024-04-25T11:19:28.818406+0000","last_undegraded":"2024-04-25T11:19:28.818406+0000","last_fullsized":"2024-04-25T11:19:28.818406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:21.523311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801895+0000","last_change":"2024-04-25T11:19:27.415292+0000","last_active":"2024-04-25T11:19:29.801895+0000","last_peered":"2024-04-25T11:19:29.801895+0000","last_clean":"2024-04-25T11:19:29.801895+0000","last_became_active":"2024-04-25T11:19:27.414270+0000","last_became_peered":"2024-04-25T11:19:27.414270+0000","last_unstale":"2024-04-25T11:19:29.801895+0000","last_undegraded":"2024-04-25T11:19:29.801895+0000","last_fullsized":"2024-04-25T11:19:29.801895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421617+0000","last_change":"2024-04-25T11:19:26.424289+0000","last_active":"2024-04-25T11:19:28.421617+0000","last_peered":"2024-04-25T11:19:28.421617+0000","last_clean":"2024-04-25T11:19:28.421617+0000","last_became_active":"2024-04-25T11:19:26.423826+0000","last_became_peered":"2024-04-25T11:19:26.423826+0000","last_unstale":"2024-04-25T11:19:28.421617+0000","last_undegraded":"2024-04-25T11:19:28.421617+0000","last_fullsized":"2024-04-25T11:19:28.421617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818628+0000","last_change":"2024-04-25T11:19:27.457615+0000","last_active":"2024-04-25T11:19:28.818628+0000","last_peered":"2024-04-25T11:19:28.818628+0000","last_clean":"2024-04-25T11:19:28.818628+0000","last_became_active":"2024-04-25T11:19:27.447354+0000","last_became_peered":"2024-04-25T11:19:27.447354+0000","last_unstale":"2024-04-25T11:19:28.818628+0000","last_undegraded":"2024-04-25T11:19:28.818628+0000","last_fullsized":"2024-04-25T11:19:28.818628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.518039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804115+0000","last_change":"2024-04-25T11:19:26.459930+0000","last_active":"2024-04-25T11:19:29.804115+0000","last_peered":"2024-04-25T11:19:29.804115+0000","last_clean":"2024-04-25T11:19:29.804115+0000","last_became_active":"2024-04-25T11:19:26.459292+0000","last_became_peered":"2024-04-25T11:19:26.459292+0000","last_unstale":"2024-04-25T11:19:29.804115+0000","last_undegraded":"2024-04-25T11:19:29.804115+0000","last_fullsized":"2024-04-25T11:19:29.804115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:08.607950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422340+0000","last_change":"2024-04-25T11:19:27.422008+0000","last_active":"2024-04-25T11:19:28.422340+0000","last_peered":"2024-04-25T11:19:28.422340+0000","last_clean":"2024-04-25T11:19:28.422340+0000","last_became_active":"2024-04-25T11:19:27.419929+0000","last_became_peered":"2024-04-25T11:19:27.419929+0000","last_unstale":"2024-04-25T11:19:28.422340+0000","last_undegraded":"2024-04-25T11:19:28.422340+0000","last_fullsized":"2024-04-25T11:19:28.422340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.431242+0000","last_change":"2024-04-25T11:19:26.435682+0000","last_active":"2024-04-25T11:19:28.431242+0000","last_peered":"2024-04-25T11:19:28.431242+0000","last_clean":"2024-04-25T11:19:28.431242+0000","last_became_active":"2024-04-25T11:19:26.435173+0000","last_became_peered":"2024-04-25T11:19:26.435173+0000","last_unstale":"2024-04-25T11:19:28.431242+0000","last_undegraded":"2024-04-25T11:19:28.431242+0000","last_fullsized":"2024-04-25T11:19:28.431242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:49.071318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792892+0000","last_change":"2024-04-25T11:19:27.446933+0000","last_active":"2024-04-25T11:19:29.792892+0000","last_peered":"2024-04-25T11:19:29.792892+0000","last_clean":"2024-04-25T11:19:29.792892+0000","last_became_active":"2024-04-25T11:19:27.446601+0000","last_became_peered":"2024-04-25T11:19:27.446601+0000","last_unstale":"2024-04-25T11:19:29.792892+0000","last_undegraded":"2024-04-25T11:19:29.792892+0000","last_fullsized":"2024-04-25T11:19:29.792892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:39.163529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433109+0000","last_change":"2024-04-25T11:19:26.451159+0000","last_active":"2024-04-25T11:19:28.433109+0000","last_peered":"2024-04-25T11:19:28.433109+0000","last_clean":"2024-04-25T11:19:28.433109+0000","last_became_active":"2024-04-25T11:19:26.450440+0000","last_became_peered":"2024-04-25T11:19:26.450440+0000","last_unstale":"2024-04-25T11:19:28.433109+0000","last_undegraded":"2024-04-25T11:19:28.433109+0000","last_fullsized":"2024-04-25T11:19:28.433109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801203+0000","last_change":"2024-04-25T11:19:27.428326+0000","last_active":"2024-04-25T11:19:29.801203+0000","last_peered":"2024-04-25T11:19:29.801203+0000","last_clean":"2024-04-25T11:19:29.801203+0000","last_became_active":"2024-04-25T11:19:27.427906+0000","last_became_peered":"2024-04-25T11:19:27.427906+0000","last_unstale":"2024-04-25T11:19:29.801203+0000","last_undegraded":"2024-04-25T11:19:29.801203+0000","last_fullsized":"2024-04-25T11:19:29.801203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801345+0000","last_change":"2024-04-25T11:19:26.447443+0000","last_active":"2024-04-25T11:19:29.801345+0000","last_peered":"2024-04-25T11:19:29.801345+0000","last_clean":"2024-04-25T11:19:29.801345+0000","last_became_active":"2024-04-25T11:19:26.439787+0000","last_became_peered":"2024-04-25T11:19:26.439787+0000","last_unstale":"2024-04-25T11:19:29.801345+0000","last_undegraded":"2024-04-25T11:19:29.801345+0000","last_fullsized":"2024-04-25T11:19:29.801345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:49.071318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801449+0000","last_change":"2024-04-25T11:19:27.422934+0000","last_active":"2024-04-25T11:19:29.801449+0000","last_peered":"2024-04-25T11:19:29.801449+0000","last_clean":"2024-04-25T11:19:29.801449+0000","last_became_active":"2024-04-25T11:19:27.421387+0000","last_became_peered":"2024-04-25T11:19:27.421387+0000","last_unstale":"2024-04-25T11:19:29.801449+0000","last_undegraded":"2024-04-25T11:19:29.801449+0000","last_fullsized":"2024-04-25T11:19:29.801449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.963336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421663+0000","last_change":"2024-04-25T11:19:26.456701+0000","last_active":"2024-04-25T11:19:28.421663+0000","last_peered":"2024-04-25T11:19:28.421663+0000","last_clean":"2024-04-25T11:19:28.421663+0000","last_became_active":"2024-04-25T11:19:26.456252+0000","last_became_peered":"2024-04-25T11:19:26.456252+0000","last_unstale":"2024-04-25T11:19:28.421663+0000","last_undegraded":"2024-04-25T11:19:28.421663+0000","last_fullsized":"2024-04-25T11:19:28.421663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791806+0000","last_change":"2024-04-25T11:19:27.435245+0000","last_active":"2024-04-25T11:19:29.791806+0000","last_peered":"2024-04-25T11:19:29.791806+0000","last_clean":"2024-04-25T11:19:29.791806+0000","last_became_active":"2024-04-25T11:19:27.434609+0000","last_became_peered":"2024-04-25T11:19:27.434609+0000","last_unstale":"2024-04-25T11:19:29.791806+0000","last_undegraded":"2024-04-25T11:19:29.791806+0000","last_fullsized":"2024-04-25T11:19:29.791806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422471+0000","last_change":"2024-04-25T11:19:26.423167+0000","last_active":"2024-04-25T11:19:28.422471+0000","last_peered":"2024-04-25T11:19:28.422471+0000","last_clean":"2024-04-25T11:19:28.422471+0000","last_became_active":"2024-04-25T11:19:26.422397+0000","last_became_peered":"2024-04-25T11:19:26.422397+0000","last_unstale":"2024-04-25T11:19:28.422471+0000","last_undegraded":"2024-04-25T11:19:28.422471+0000","last_fullsized":"2024-04-25T11:19:28.422471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.048030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792180+0000","last_change":"2024-04-25T11:19:27.426394+0000","last_active":"2024-04-25T11:19:29.792180+0000","last_peered":"2024-04-25T11:19:29.792180+0000","last_clean":"2024-04-25T11:19:29.792180+0000","last_became_active":"2024-04-25T11:19:27.424504+0000","last_became_peered":"2024-04-25T11:19:27.424504+0000","last_unstale":"2024-04-25T11:19:29.792180+0000","last_undegraded":"2024-04-25T11:19:29.792180+0000","last_fullsized":"2024-04-25T11:19:29.792180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.850268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803567+0000","last_change":"2024-04-25T11:19:26.455784+0000","last_active":"2024-04-25T11:19:29.803567+0000","last_peered":"2024-04-25T11:19:29.803567+0000","last_clean":"2024-04-25T11:19:29.803567+0000","last_became_active":"2024-04-25T11:19:26.455125+0000","last_became_peered":"2024-04-25T11:19:26.455125+0000","last_unstale":"2024-04-25T11:19:29.803567+0000","last_undegraded":"2024-04-25T11:19:29.803567+0000","last_fullsized":"2024-04-25T11:19:29.803567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:40.075026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433197+0000","last_change":"2024-04-25T11:19:27.455131+0000","last_active":"2024-04-25T11:19:28.433197+0000","last_peered":"2024-04-25T11:19:28.433197+0000","last_clean":"2024-04-25T11:19:28.433197+0000","last_became_active":"2024-04-25T11:19:27.454732+0000","last_became_peered":"2024-04-25T11:19:27.454732+0000","last_unstale":"2024-04-25T11:19:28.433197+0000","last_undegraded":"2024-04-25T11:19:28.433197+0000","last_fullsized":"2024-04-25T11:19:28.433197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818302+0000","last_change":"2024-04-25T11:19:27.450900+0000","last_active":"2024-04-25T11:19:28.818302+0000","last_peered":"2024-04-25T11:19:28.818302+0000","last_clean":"2024-04-25T11:19:28.818302+0000","last_became_active":"2024-04-25T11:19:27.447933+0000","last_became_peered":"2024-04-25T11:19:27.447933+0000","last_unstale":"2024-04-25T11:19:28.818302+0000","last_undegraded":"2024-04-25T11:19:28.818302+0000","last_fullsized":"2024-04-25T11:19:28.818302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:03.621864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802750+0000","last_change":"2024-04-25T11:19:26.456557+0000","last_active":"2024-04-25T11:19:29.802750+0000","last_peered":"2024-04-25T11:19:29.802750+0000","last_clean":"2024-04-25T11:19:29.802750+0000","last_became_active":"2024-04-25T11:19:26.455740+0000","last_became_peered":"2024-04-25T11:19:26.455740+0000","last_unstale":"2024-04-25T11:19:29.802750+0000","last_undegraded":"2024-04-25T11:19:29.802750+0000","last_fullsized":"2024-04-25T11:19:29.802750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:12.867967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818605+0000","last_change":"2024-04-25T11:19:27.440111+0000","last_active":"2024-04-25T11:19:28.818605+0000","last_peered":"2024-04-25T11:19:28.818605+0000","last_clean":"2024-04-25T11:19:28.818605+0000","last_became_active":"2024-04-25T11:19:27.430503+0000","last_became_peered":"2024-04-25T11:19:27.430503+0000","last_unstale":"2024-04-25T11:19:28.818605+0000","last_undegraded":"2024-04-25T11:19:28.818605+0000","last_fullsized":"2024-04-25T11:19:28.818605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:45.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803868+0000","last_change":"2024-04-25T11:19:26.457866+0000","last_active":"2024-04-25T11:19:29.803868+0000","last_peered":"2024-04-25T11:19:29.803868+0000","last_clean":"2024-04-25T11:19:29.803868+0000","last_became_active":"2024-04-25T11:19:26.457223+0000","last_became_peered":"2024-04-25T11:19:26.457223+0000","last_unstale":"2024-04-25T11:19:29.803868+0000","last_undegraded":"2024-04-25T11:19:29.803868+0000","last_fullsized":"2024-04-25T11:19:29.803868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.048030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801205+0000","last_change":"2024-04-25T11:19:27.448305+0000","last_active":"2024-04-25T11:19:29.801205+0000","last_peered":"2024-04-25T11:19:29.801205+0000","last_clean":"2024-04-25T11:19:29.801205+0000","last_became_active":"2024-04-25T11:19:27.447940+0000","last_became_peered":"2024-04-25T11:19:27.447940+0000","last_unstale":"2024-04-25T11:19:29.801205+0000","last_undegraded":"2024-04-25T11:19:29.801205+0000","last_fullsized":"2024-04-25T11:19:29.801205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792735+0000","last_change":"2024-04-25T11:19:26.425844+0000","last_active":"2024-04-25T11:19:29.792735+0000","last_peered":"2024-04-25T11:19:29.792735+0000","last_clean":"2024-04-25T11:19:29.792735+0000","last_became_active":"2024-04-25T11:19:26.425443+0000","last_became_peered":"2024-04-25T11:19:26.425443+0000","last_unstale":"2024-04-25T11:19:29.792735+0000","last_undegraded":"2024-04-25T11:19:29.792735+0000","last_fullsized":"2024-04-25T11:19:29.792735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792425+0000","last_change":"2024-04-25T11:19:27.434918+0000","last_active":"2024-04-25T11:19:29.792425+0000","last_peered":"2024-04-25T11:19:29.792425+0000","last_clean":"2024-04-25T11:19:29.792425+0000","last_became_active":"2024-04-25T11:19:27.434444+0000","last_became_peered":"2024-04-25T11:19:27.434444+0000","last_unstale":"2024-04-25T11:19:29.792425+0000","last_undegraded":"2024-04-25T11:19:29.792425+0000","last_fullsized":"2024-04-25T11:19:29.792425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.398422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800979+0000","last_change":"2024-04-25T11:19:26.448774+0000","last_active":"2024-04-25T11:19:29.800979+0000","last_peered":"2024-04-25T11:19:29.800979+0000","last_clean":"2024-04-25T11:19:29.800979+0000","last_became_active":"2024-04-25T11:19:26.448304+0000","last_became_peered":"2024-04-25T11:19:26.448304+0000","last_unstale":"2024-04-25T11:19:29.800979+0000","last_undegraded":"2024-04-25T11:19:29.800979+0000","last_fullsized":"2024-04-25T11:19:29.800979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:19:36.040 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:19:36.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.217+0000 7f5862f1a640 1 Processor -- start 2024-04-25T11:19:36.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.217+0000 7f5862f1a640 1 -- start start 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f5862f1a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c0d5700 0x7f585c0d5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f5862f1a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585c0d6050 con 0x7f585c15fda0 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f5862f1a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585c1601b0 con 0x7f585c0d4480 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f5862f1a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f585c163db0 con 0x7f585c0d5700 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f5860cb6640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f585c15fda0 0x7f585c0b8c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54256/0 (socket says 172.21.15.106:54256) 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f5860cb6640 1 -- 172.21.15.106:0/406237596 learned_addr learned my addr 172.21.15.106:0/406237596 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585bfff640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c0d5700 0x7f585c0d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963728537 0 0) 0x7f584c001120 con 0x7f585c15fda0 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5844003530 con 0x7f585c15fda0 2024-04-25T11:19:36.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 7664948 0 0) 0x7f5844003530 con 0x7f585c15fda0 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c0d5700 msgr2=0x7f585c0d5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c0d5700 0x7f585c0d5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 >> v1:172.21.15.134:6789/0 conn(0x7f585c0d4480 legacy=0x7f585c0d4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c164090 con 0x7f585c15fda0 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f584c001850 con 0x7f585c15fda0 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 >> v1:172.21.15.106:6789/0 conn(0x7f585c15fda0 legacy=0x7f585c0b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58440039f0 0x7f5844003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5844004810 0x7f5844004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c0d5700 0x7f5844005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f585c163db0 con 0x7f5844004810 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f584400bb30 con 0x7f58440039f0 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f584400bd30 con 0x7f585c0d5700 2024-04-25T11:19:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585bfff640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5844004810 0x7f5844004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585bfff640 1 -- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c0d5700 msgr2=0x7f5844005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585bfff640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c0d5700 0x7f5844005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585bfff640 1 -- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58440039f0 msgr2=0x7f5844003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585bfff640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58440039f0 0x7f5844003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.221+0000 7f585bfff640 1 -- 172.21.15.106:0/406237596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5844003530 con 0x7f5844004810 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f585bfff640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5844004810 0x7f5844004c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5850009ab0 tx=0x7f585001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12a9c544453d4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500b2070 con 0x7f5844004810 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58500a55c0 con 0x7f5844004810 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f585b7fe640 1 -- 172.21.15.106:0/406237596 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500096b0 con 0x7f5844004810 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f5862f1a640 1 -- 172.21.15.106:0/406237596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5844004810 msgr2=0x7f5844004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f5862f1a640 1 --2- 172.21.15.106:0/406237596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5844004810 0x7f5844004c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5850009ab0 tx=0x7f585001c9d0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f58614b7640 1 -- 172.21.15.106:0/406237596 reap_dead start 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f5862f1a640 1 -- 172.21.15.106:0/406237596 shutdown_connections 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f5862f1a640 1 -- 172.21.15.106:0/406237596 >> 172.21.15.106:0/406237596 conn(0x7f585c06d8e0 msgr2=0x7f585c06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f5862f1a640 1 -- 172.21.15.106:0/406237596 shutdown_connections 2024-04-25T11:19:36.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f5862f1a640 1 -- 172.21.15.106:0/406237596 wait complete. 2024-04-25T11:19:36.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f5862f1a640 1 Processor -- start 2024-04-25T11:19:36.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.225+0000 7f5862f1a640 1 -- start start 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5844004810 0x7f585c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f585c14b990 0x7f585c14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c14c2e0 0x7f585c14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f585c0d4fa0 con 0x7f585c14b990 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f585c164650 con 0x7f5844004810 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f585c0d5120 con 0x7f585c14c2e0 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f585c14b990 0x7f585c14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f585c14b990 0x7f585c14bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44500/0 (socket says 172.21.15.106:44500) 2024-04-25T11:19:36.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 -- 172.21.15.106:0/2059384749 learned_addr learned my addr 172.21.15.106:0/2059384749 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:36.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5860cb6640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5844004810 0x7f585c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f58614b7640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c14c2e0 0x7f585c14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 -- 172.21.15.106:0/2059384749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c14c2e0 msgr2=0x7f585c14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c14c2e0 0x7f585c14cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 -- 172.21.15.106:0/2059384749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5844004810 msgr2=0x7f585c0af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5844004810 0x7f585c0af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 -- 172.21.15.106:0/2059384749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c162b30 con 0x7f585c14b990 2024-04-25T11:19:36.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f585bfff640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f585c14b990 0x7f585c14bda0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5850009ab0 tx=0x7f5844005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f58597fa640 1 -- 172.21.15.106:0/2059384749 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500b2040 con 0x7f585c14b990 2024-04-25T11:19:36.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f58597fa640 1 -- 172.21.15.106:0/2059384749 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58500a5df0 con 0x7f585c14b990 2024-04-25T11:19:36.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c14aeb0 con 0x7f585c14b990 2024-04-25T11:19:36.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f58597fa640 1 -- 172.21.15.106:0/2059384749 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500096b0 con 0x7f585c14b990 2024-04-25T11:19:36.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c14ca10 con 0x7f585c14b990 2024-04-25T11:19:36.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c0b4fd0 con 0x7f585c14b990 2024-04-25T11:19:36.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.229+0000 7f58597fa640 1 -- 172.21.15.106:0/2059384749 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f58500c0020 con 0x7f585c14b990 2024-04-25T11:19:36.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.233+0000 7f58597fa640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f584806f430 0x7f5848071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.233+0000 7f58597fa640 1 -- 172.21.15.106:0/2059384749 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5850129a00 con 0x7f585c14b990 2024-04-25T11:19:36.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.233+0000 7f5860cb6640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f584806f430 0x7f5848071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.233+0000 7f5860cb6640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f584806f430 0x7f5848071940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f584c004880 tx=0x7f584c0016d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.237+0000 7f58597fa640 1 -- 172.21.15.106:0/2059384749 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58500c7020 con 0x7f585c14b990 2024-04-25T11:19:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.381+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f585c0d3b70 con 0x7f584806f430 2024-04-25T11:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.389+0000 7f58597fa640 1 -- 172.21.15.106:0/2059384749 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f585c0d3b70 con 0x7f584806f430 2024-04-25T11:19:36.389 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:36.391 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:19:36.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f584806f430 msgr2=0x7f5848071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f584806f430 0x7f5848071940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f584c004880 tx=0x7f584c0016d0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f585c14b990 msgr2=0x7f585c14bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f585c14b990 0x7f585c14bda0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5850009ab0 tx=0x7f5844005840 comp rx=0 tx=0).stop 2024-04-25T11:19:36.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 shutdown_connections 2024-04-25T11:19:36.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f585c14c2e0 0x7f585c14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f584806f430 0x7f5848071940 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f585c14b990 0x7f585c14bda0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 --2- 172.21.15.106:0/2059384749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5844004810 0x7f585c0af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 >> 172.21.15.106:0/2059384749 conn(0x7f585c06d8e0 msgr2=0x7f585c0badc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 shutdown_connections 2024-04-25T11:19:36.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.393+0000 7f5862f1a640 1 -- 172.21.15.106:0/2059384749 wait complete. 2024-04-25T11:19:36.409 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T11:19:36.093674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.700113"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804327+0000","last_change":"2024-04-25T11:19:27.448704+0000","last_active":"2024-04-25T11:19:29.804327+0000","last_peered":"2024-04-25T11:19:29.804327+0000","last_clean":"2024-04-25T11:19:29.804327+0000","last_became_active":"2024-04-25T11:19:27.447782+0000","last_became_peered":"2024-04-25T11:19:27.447782+0000","last_unstale":"2024-04-25T11:19:29.804327+0000","last_undegraded":"2024-04-25T11:19:29.804327+0000","last_fullsized":"2024-04-25T11:19:29.804327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.297417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422162+0000","last_change":"2024-04-25T11:19:27.424499+0000","last_active":"2024-04-25T11:19:28.422162+0000","last_peered":"2024-04-25T11:19:28.422162+0000","last_clean":"2024-04-25T11:19:28.422162+0000","last_became_active":"2024-04-25T11:19:27.424122+0000","last_became_peered":"2024-04-25T11:19:27.424122+0000","last_unstale":"2024-04-25T11:19:28.422162+0000","last_undegraded":"2024-04-25T11:19:28.422162+0000","last_fullsized":"2024-04-25T11:19:28.422162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800964+0000","last_change":"2024-04-25T11:19:27.422128+0000","last_active":"2024-04-25T11:19:29.800964+0000","last_peered":"2024-04-25T11:19:29.800964+0000","last_clean":"2024-04-25T11:19:29.800964+0000","last_became_active":"2024-04-25T11:19:27.421683+0000","last_became_peered":"2024-04-25T11:19:27.421683+0000","last_unstale":"2024-04-25T11:19:29.800964+0000","last_undegraded":"2024-04-25T11:19:29.800964+0000","last_fullsized":"2024-04-25T11:19:29.800964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818544+0000","last_change":"2024-04-25T11:19:27.439794+0000","last_active":"2024-04-25T11:19:28.818544+0000","last_peered":"2024-04-25T11:19:28.818544+0000","last_clean":"2024-04-25T11:19:28.818544+0000","last_became_active":"2024-04-25T11:19:27.439092+0000","last_became_peered":"2024-04-25T11:19:27.439092+0000","last_unstale":"2024-04-25T11:19:28.818544+0000","last_undegraded":"2024-04-25T11:19:28.818544+0000","last_fullsized":"2024-04-25T11:19:28.818544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:52.757499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803029+0000","last_change":"2024-04-25T11:19:27.447315+0000","last_active":"2024-04-25T11:19:29.803029+0000","last_peered":"2024-04-25T11:19:29.803029+0000","last_clean":"2024-04-25T11:19:29.803029+0000","last_became_active":"2024-04-25T11:19:27.446274+0000","last_became_peered":"2024-04-25T11:19:27.446274+0000","last_unstale":"2024-04-25T11:19:29.803029+0000","last_undegraded":"2024-04-25T11:19:29.803029+0000","last_fullsized":"2024-04-25T11:19:29.803029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:01.196607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817544+0000","last_change":"2024-04-25T11:19:27.446758+0000","last_active":"2024-04-25T11:19:28.817544+0000","last_peered":"2024-04-25T11:19:28.817544+0000","last_clean":"2024-04-25T11:19:28.817544+0000","last_became_active":"2024-04-25T11:19:27.440252+0000","last_became_peered":"2024-04-25T11:19:27.440252+0000","last_unstale":"2024-04-25T11:19:28.817544+0000","last_undegraded":"2024-04-25T11:19:28.817544+0000","last_fullsized":"2024-04-25T11:19:28.817544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:08.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803004+0000","last_change":"2024-04-25T11:19:27.442783+0000","last_active":"2024-04-25T11:19:29.803004+0000","last_peered":"2024-04-25T11:19:29.803004+0000","last_clean":"2024-04-25T11:19:29.803004+0000","last_became_active":"2024-04-25T11:19:27.440913+0000","last_became_peered":"2024-04-25T11:19:27.440913+0000","last_unstale":"2024-04-25T11:19:29.803004+0000","last_undegraded":"2024-04-25T11:19:29.803004+0000","last_fullsized":"2024-04-25T11:19:29.803004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.329120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791320+0000","last_change":"2024-04-25T11:19:27.441574+0000","last_active":"2024-04-25T11:19:29.791320+0000","last_peered":"2024-04-25T11:19:29.791320+0000","last_clean":"2024-04-25T11:19:29.791320+0000","last_became_active":"2024-04-25T11:19:27.441265+0000","last_became_peered":"2024-04-25T11:19:27.441265+0000","last_unstale":"2024-04-25T11:19:29.791320+0000","last_undegraded":"2024-04-25T11:19:29.791320+0000","last_fullsized":"2024-04-25T11:19:29.791320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804555+0000","last_change":"2024-04-25T11:19:27.459170+0000","last_active":"2024-04-25T11:19:29.804555+0000","last_peered":"2024-04-25T11:19:29.804555+0000","last_clean":"2024-04-25T11:19:29.804555+0000","last_became_active":"2024-04-25T11:19:27.458755+0000","last_became_peered":"2024-04-25T11:19:27.458755+0000","last_unstale":"2024-04-25T11:19:29.804555+0000","last_undegraded":"2024-04-25T11:19:29.804555+0000","last_fullsized":"2024-04-25T11:19:29.804555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:52.797941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433030+0000","last_change":"2024-04-25T11:19:27.456078+0000","last_active":"2024-04-25T11:19:28.433030+0000","last_peered":"2024-04-25T11:19:28.433030+0000","last_clean":"2024-04-25T11:19:28.433030+0000","last_became_active":"2024-04-25T11:19:27.455302+0000","last_became_peered":"2024-04-25T11:19:27.455302+0000","last_unstale":"2024-04-25T11:19:28.433030+0000","last_undegraded":"2024-04-25T11:19:28.433030+0000","last_fullsized":"2024-04-25T11:19:28.433030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432803+0000","last_change":"2024-04-25T11:19:27.459132+0000","last_active":"2024-04-25T11:19:28.432803+0000","last_peered":"2024-04-25T11:19:28.432803+0000","last_clean":"2024-04-25T11:19:28.432803+0000","last_became_active":"2024-04-25T11:19:27.458820+0000","last_became_peered":"2024-04-25T11:19:27.458820+0000","last_unstale":"2024-04-25T11:19:28.432803+0000","last_undegraded":"2024-04-25T11:19:28.432803+0000","last_fullsized":"2024-04-25T11:19:28.432803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.067341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802226+0000","last_change":"2024-04-25T11:19:27.439470+0000","last_active":"2024-04-25T11:19:29.802226+0000","last_peered":"2024-04-25T11:19:29.802226+0000","last_clean":"2024-04-25T11:19:29.802226+0000","last_became_active":"2024-04-25T11:19:27.439141+0000","last_became_peered":"2024-04-25T11:19:27.439141+0000","last_unstale":"2024-04-25T11:19:29.802226+0000","last_undegraded":"2024-04-25T11:19:29.802226+0000","last_fullsized":"2024-04-25T11:19:29.802226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.059005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791949+0000","last_change":"2024-04-25T11:19:27.433821+0000","last_active":"2024-04-25T11:19:29.791949+0000","last_peered":"2024-04-25T11:19:29.791949+0000","last_clean":"2024-04-25T11:19:29.791949+0000","last_became_active":"2024-04-25T11:19:27.433427+0000","last_became_peered":"2024-04-25T11:19:27.433427+0000","last_unstale":"2024-04-25T11:19:29.791949+0000","last_undegraded":"2024-04-25T11:19:29.791949+0000","last_fullsized":"2024-04-25T11:19:29.791949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800437+0000","last_change":"2024-04-25T11:19:27.428346+0000","last_active":"2024-04-25T11:19:29.800437+0000","last_peered":"2024-04-25T11:19:29.800437+0000","last_clean":"2024-04-25T11:19:29.800437+0000","last_became_active":"2024-04-25T11:19:27.428006+0000","last_became_peered":"2024-04-25T11:19:27.428006+0000","last_unstale":"2024-04-25T11:19:29.800437+0000","last_undegraded":"2024-04-25T11:19:29.800437+0000","last_fullsized":"2024-04-25T11:19:29.800437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:41.086001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421393+0000","last_change":"2024-04-25T11:19:27.428566+0000","last_active":"2024-04-25T11:19:28.421393+0000","last_peered":"2024-04-25T11:19:28.421393+0000","last_clean":"2024-04-25T11:19:28.421393+0000","last_became_active":"2024-04-25T11:19:27.427282+0000","last_became_peered":"2024-04-25T11:19:27.427282+0000","last_unstale":"2024-04-25T11:19:28.421393+0000","last_undegraded":"2024-04-25T11:19:28.421393+0000","last_fullsized":"2024-04-25T11:19:28.421393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817322+0000","last_change":"2024-04-25T11:19:27.448380+0000","last_active":"2024-04-25T11:19:28.817322+0000","last_peered":"2024-04-25T11:19:28.817322+0000","last_clean":"2024-04-25T11:19:28.817322+0000","last_became_active":"2024-04-25T11:19:27.447971+0000","last_became_peered":"2024-04-25T11:19:27.447971+0000","last_unstale":"2024-04-25T11:19:28.817322+0000","last_undegraded":"2024-04-25T11:19:28.817322+0000","last_fullsized":"2024-04-25T11:19:28.817322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.850268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818713+0000","last_change":"2024-04-25T11:19:27.452389+0000","last_active":"2024-04-25T11:19:28.818713+0000","last_peered":"2024-04-25T11:19:28.818713+0000","last_clean":"2024-04-25T11:19:28.818713+0000","last_became_active":"2024-04-25T11:19:27.451910+0000","last_became_peered":"2024-04-25T11:19:27.451910+0000","last_unstale":"2024-04-25T11:19:28.818713+0000","last_undegraded":"2024-04-25T11:19:28.818713+0000","last_fullsized":"2024-04-25T11:19:28.818713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424567+0000","last_change":"2024-04-25T11:19:27.437476+0000","last_active":"2024-04-25T11:19:28.424567+0000","last_peered":"2024-04-25T11:19:28.424567+0000","last_clean":"2024-04-25T11:19:28.424567+0000","last_became_active":"2024-04-25T11:19:27.436712+0000","last_became_peered":"2024-04-25T11:19:27.436712+0000","last_unstale":"2024-04-25T11:19:28.424567+0000","last_undegraded":"2024-04-25T11:19:28.424567+0000","last_fullsized":"2024-04-25T11:19:28.424567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804153+0000","last_change":"2024-04-25T11:19:27.445559+0000","last_active":"2024-04-25T11:19:29.804153+0000","last_peered":"2024-04-25T11:19:29.804153+0000","last_clean":"2024-04-25T11:19:29.804153+0000","last_became_active":"2024-04-25T11:19:27.445058+0000","last_became_peered":"2024-04-25T11:19:27.445058+0000","last_unstale":"2024-04-25T11:19:29.804153+0000","last_undegraded":"2024-04-25T11:19:29.804153+0000","last_fullsized":"2024-04-25T11:19:29.804153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432846+0000","last_change":"2024-04-25T11:19:27.441427+0000","last_active":"2024-04-25T11:19:28.432846+0000","last_peered":"2024-04-25T11:19:28.432846+0000","last_clean":"2024-04-25T11:19:28.432846+0000","last_became_active":"2024-04-25T11:19:27.441087+0000","last_became_peered":"2024-04-25T11:19:27.441087+0000","last_unstale":"2024-04-25T11:19:28.432846+0000","last_undegraded":"2024-04-25T11:19:28.432846+0000","last_fullsized":"2024-04-25T11:19:28.432846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801155+0000","last_change":"2024-04-25T11:19:27.445781+0000","last_active":"2024-04-25T11:19:29.801155+0000","last_peered":"2024-04-25T11:19:29.801155+0000","last_clean":"2024-04-25T11:19:29.801155+0000","last_became_active":"2024-04-25T11:19:27.445240+0000","last_became_peered":"2024-04-25T11:19:27.445240+0000","last_unstale":"2024-04-25T11:19:29.801155+0000","last_undegraded":"2024-04-25T11:19:29.801155+0000","last_fullsized":"2024-04-25T11:19:29.801155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817748+0000","last_change":"2024-04-25T11:19:27.453395+0000","last_active":"2024-04-25T11:19:28.817748+0000","last_peered":"2024-04-25T11:19:28.817748+0000","last_clean":"2024-04-25T11:19:28.817748+0000","last_became_active":"2024-04-25T11:19:27.452942+0000","last_became_peered":"2024-04-25T11:19:27.452942+0000","last_unstale":"2024-04-25T11:19:28.817748+0000","last_undegraded":"2024-04-25T11:19:28.817748+0000","last_fullsized":"2024-04-25T11:19:28.817748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:39.163529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791445+0000","last_change":"2024-04-25T11:19:27.427559+0000","last_active":"2024-04-25T11:19:29.791445+0000","last_peered":"2024-04-25T11:19:29.791445+0000","last_clean":"2024-04-25T11:19:29.791445+0000","last_became_active":"2024-04-25T11:19:27.421099+0000","last_became_peered":"2024-04-25T11:19:27.421099+0000","last_unstale":"2024-04-25T11:19:29.791445+0000","last_undegraded":"2024-04-25T11:19:29.791445+0000","last_fullsized":"2024-04-25T11:19:29.791445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:10.369182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817459+0000","last_change":"2024-04-25T11:19:27.440162+0000","last_active":"2024-04-25T11:19:28.817459+0000","last_peered":"2024-04-25T11:19:28.817459+0000","last_clean":"2024-04-25T11:19:28.817459+0000","last_became_active":"2024-04-25T11:19:27.439384+0000","last_became_peered":"2024-04-25T11:19:27.439384+0000","last_unstale":"2024-04-25T11:19:28.817459+0000","last_undegraded":"2024-04-25T11:19:28.817459+0000","last_fullsized":"2024-04-25T11:19:28.817459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.398422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792627+0000","last_change":"2024-04-25T11:19:27.438911+0000","last_active":"2024-04-25T11:19:29.792627+0000","last_peered":"2024-04-25T11:19:29.792627+0000","last_clean":"2024-04-25T11:19:29.792627+0000","last_became_active":"2024-04-25T11:19:27.437950+0000","last_became_peered":"2024-04-25T11:19:27.437950+0000","last_unstale":"2024-04-25T11:19:29.792627+0000","last_undegraded":"2024-04-25T11:19:29.792627+0000","last_fullsized":"2024-04-25T11:19:29.792627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:08.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433178+0000","last_change":"2024-04-25T11:19:27.447613+0000","last_active":"2024-04-25T11:19:28.433178+0000","last_peered":"2024-04-25T11:19:28.433178+0000","last_clean":"2024-04-25T11:19:28.433178+0000","last_became_active":"2024-04-25T11:19:27.447314+0000","last_became_peered":"2024-04-25T11:19:27.447314+0000","last_unstale":"2024-04-25T11:19:28.433178+0000","last_undegraded":"2024-04-25T11:19:28.433178+0000","last_fullsized":"2024-04-25T11:19:28.433178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421840+0000","last_change":"2024-04-25T11:19:27.436767+0000","last_active":"2024-04-25T11:19:28.421840+0000","last_peered":"2024-04-25T11:19:28.421840+0000","last_clean":"2024-04-25T11:19:28.421840+0000","last_became_active":"2024-04-25T11:19:27.436392+0000","last_became_peered":"2024-04-25T11:19:27.436392+0000","last_unstale":"2024-04-25T11:19:28.421840+0000","last_undegraded":"2024-04-25T11:19:28.421840+0000","last_fullsized":"2024-04-25T11:19:28.421840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792127+0000","last_change":"2024-04-25T11:19:27.438536+0000","last_active":"2024-04-25T11:19:29.792127+0000","last_peered":"2024-04-25T11:19:29.792127+0000","last_clean":"2024-04-25T11:19:29.792127+0000","last_became_active":"2024-04-25T11:19:27.438198+0000","last_became_peered":"2024-04-25T11:19:27.438198+0000","last_unstale":"2024-04-25T11:19:29.792127+0000","last_undegraded":"2024-04-25T11:19:29.792127+0000","last_fullsized":"2024-04-25T11:19:29.792127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.942052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801440+0000","last_change":"2024-04-25T11:19:27.425325+0000","last_active":"2024-04-25T11:19:29.801440+0000","last_peered":"2024-04-25T11:19:29.801440+0000","last_clean":"2024-04-25T11:19:29.801440+0000","last_became_active":"2024-04-25T11:19:27.423815+0000","last_became_peered":"2024-04-25T11:19:27.423815+0000","last_unstale":"2024-04-25T11:19:29.801440+0000","last_undegraded":"2024-04-25T11:19:29.801440+0000","last_fullsized":"2024-04-25T11:19:29.801440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.942052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818101+0000","last_change":"2024-04-25T11:19:27.447151+0000","last_active":"2024-04-25T11:19:28.818101+0000","last_peered":"2024-04-25T11:19:28.818101+0000","last_clean":"2024-04-25T11:19:28.818101+0000","last_became_active":"2024-04-25T11:19:27.446193+0000","last_became_peered":"2024-04-25T11:19:27.446193+0000","last_unstale":"2024-04-25T11:19:28.818101+0000","last_undegraded":"2024-04-25T11:19:28.818101+0000","last_fullsized":"2024-04-25T11:19:28.818101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:01.196607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424335+0000","last_change":"2024-04-25T11:19:27.440245+0000","last_active":"2024-04-25T11:19:28.424335+0000","last_peered":"2024-04-25T11:19:28.424335+0000","last_clean":"2024-04-25T11:19:28.424335+0000","last_became_active":"2024-04-25T11:19:27.439755+0000","last_became_peered":"2024-04-25T11:19:27.439755+0000","last_unstale":"2024-04-25T11:19:28.424335+0000","last_undegraded":"2024-04-25T11:19:28.424335+0000","last_fullsized":"2024-04-25T11:19:28.424335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433384+0000","last_change":"2024-04-25T11:19:27.433866+0000","last_active":"2024-04-25T11:19:28.433384+0000","last_peered":"2024-04-25T11:19:28.433384+0000","last_clean":"2024-04-25T11:19:28.433384+0000","last_became_active":"2024-04-25T11:19:27.423031+0000","last_became_peered":"2024-04-25T11:19:27.423031+0000","last_unstale":"2024-04-25T11:19:28.433384+0000","last_undegraded":"2024-04-25T11:19:28.433384+0000","last_fullsized":"2024-04-25T11:19:28.433384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801476+0000","last_change":"2024-04-25T11:19:27.427233+0000","last_active":"2024-04-25T11:19:29.801476+0000","last_peered":"2024-04-25T11:19:29.801476+0000","last_clean":"2024-04-25T11:19:29.801476+0000","last_became_active":"2024-04-25T11:19:27.426438+0000","last_became_peered":"2024-04-25T11:19:27.426438+0000","last_unstale":"2024-04-25T11:19:29.801476+0000","last_undegraded":"2024-04-25T11:19:29.801476+0000","last_fullsized":"2024-04-25T11:19:29.801476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:25.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792171+0000","last_change":"2024-04-25T11:19:26.448872+0000","last_active":"2024-04-25T11:19:29.792171+0000","last_peered":"2024-04-25T11:19:29.792171+0000","last_clean":"2024-04-25T11:19:29.792171+0000","last_became_active":"2024-04-25T11:19:26.448492+0000","last_became_peered":"2024-04-25T11:19:26.448492+0000","last_unstale":"2024-04-25T11:19:29.792171+0000","last_undegraded":"2024-04-25T11:19:29.792171+0000","last_fullsized":"2024-04-25T11:19:29.792171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801354+0000","last_change":"2024-04-25T11:19:27.424636+0000","last_active":"2024-04-25T11:19:29.801354+0000","last_peered":"2024-04-25T11:19:29.801354+0000","last_clean":"2024-04-25T11:19:29.801354+0000","last_became_active":"2024-04-25T11:19:27.422364+0000","last_became_peered":"2024-04-25T11:19:27.422364+0000","last_unstale":"2024-04-25T11:19:29.801354+0000","last_undegraded":"2024-04-25T11:19:29.801354+0000","last_fullsized":"2024-04-25T11:19:29.801354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432509+0000","last_change":"2024-04-25T11:19:26.446971+0000","last_active":"2024-04-25T11:19:28.432509+0000","last_peered":"2024-04-25T11:19:28.432509+0000","last_clean":"2024-04-25T11:19:28.432509+0000","last_became_active":"2024-04-25T11:19:26.445751+0000","last_became_peered":"2024-04-25T11:19:26.445751+0000","last_unstale":"2024-04-25T11:19:28.432509+0000","last_undegraded":"2024-04-25T11:19:28.432509+0000","last_fullsized":"2024-04-25T11:19:28.432509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791338+0000","last_change":"2024-04-25T11:19:26.430729+0000","last_active":"2024-04-25T11:19:29.791338+0000","last_peered":"2024-04-25T11:19:29.791338+0000","last_clean":"2024-04-25T11:19:29.791338+0000","last_became_active":"2024-04-25T11:19:26.430240+0000","last_became_peered":"2024-04-25T11:19:26.430240+0000","last_unstale":"2024-04-25T11:19:29.791338+0000","last_undegraded":"2024-04-25T11:19:29.791338+0000","last_fullsized":"2024-04-25T11:19:29.791338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:10.270275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791441+0000","last_change":"2024-04-25T11:19:27.425985+0000","last_active":"2024-04-25T11:19:29.791441+0000","last_peered":"2024-04-25T11:19:29.791441+0000","last_clean":"2024-04-25T11:19:29.791441+0000","last_became_active":"2024-04-25T11:19:27.424091+0000","last_became_peered":"2024-04-25T11:19:27.424091+0000","last_unstale":"2024-04-25T11:19:29.791441+0000","last_undegraded":"2024-04-25T11:19:29.791441+0000","last_fullsized":"2024-04-25T11:19:29.791441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817641+0000","last_change":"2024-04-25T11:19:26.446399+0000","last_active":"2024-04-25T11:19:28.817641+0000","last_peered":"2024-04-25T11:19:28.817641+0000","last_clean":"2024-04-25T11:19:28.817641+0000","last_became_active":"2024-04-25T11:19:26.445502+0000","last_became_peered":"2024-04-25T11:19:26.445502+0000","last_unstale":"2024-04-25T11:19:28.817641+0000","last_undegraded":"2024-04-25T11:19:28.817641+0000","last_fullsized":"2024-04-25T11:19:28.817641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421333+0000","last_change":"2024-04-25T11:19:27.435289+0000","last_active":"2024-04-25T11:19:28.421333+0000","last_peered":"2024-04-25T11:19:28.421333+0000","last_clean":"2024-04-25T11:19:28.421333+0000","last_became_active":"2024-04-25T11:19:27.434888+0000","last_became_peered":"2024-04-25T11:19:27.434888+0000","last_unstale":"2024-04-25T11:19:28.421333+0000","last_undegraded":"2024-04-25T11:19:28.421333+0000","last_fullsized":"2024-04-25T11:19:28.421333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432966+0000","last_change":"2024-04-25T11:19:26.447470+0000","last_active":"2024-04-25T11:19:28.432966+0000","last_peered":"2024-04-25T11:19:28.432966+0000","last_clean":"2024-04-25T11:19:28.432966+0000","last_became_active":"2024-04-25T11:19:26.445469+0000","last_became_peered":"2024-04-25T11:19:26.445469+0000","last_unstale":"2024-04-25T11:19:28.432966+0000","last_undegraded":"2024-04-25T11:19:28.432966+0000","last_fullsized":"2024-04-25T11:19:28.432966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818172+0000","last_change":"2024-04-25T11:19:27.454986+0000","last_active":"2024-04-25T11:19:28.818172+0000","last_peered":"2024-04-25T11:19:28.818172+0000","last_clean":"2024-04-25T11:19:28.818172+0000","last_became_active":"2024-04-25T11:19:27.448543+0000","last_became_peered":"2024-04-25T11:19:27.448543+0000","last_unstale":"2024-04-25T11:19:28.818172+0000","last_undegraded":"2024-04-25T11:19:28.818172+0000","last_fullsized":"2024-04-25T11:19:28.818172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.329120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818061+0000","last_change":"2024-04-25T11:19:26.446709+0000","last_active":"2024-04-25T11:19:28.818061+0000","last_peered":"2024-04-25T11:19:28.818061+0000","last_clean":"2024-04-25T11:19:28.818061+0000","last_became_active":"2024-04-25T11:19:26.445742+0000","last_became_peered":"2024-04-25T11:19:26.445742+0000","last_unstale":"2024-04-25T11:19:28.818061+0000","last_undegraded":"2024-04-25T11:19:28.818061+0000","last_fullsized":"2024-04-25T11:19:28.818061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800839+0000","last_change":"2024-04-25T11:19:27.431751+0000","last_active":"2024-04-25T11:19:29.800839+0000","last_peered":"2024-04-25T11:19:29.800839+0000","last_clean":"2024-04-25T11:19:29.800839+0000","last_became_active":"2024-04-25T11:19:27.431063+0000","last_became_peered":"2024-04-25T11:19:27.431063+0000","last_unstale":"2024-04-25T11:19:29.800839+0000","last_undegraded":"2024-04-25T11:19:29.800839+0000","last_fullsized":"2024-04-25T11:19:29.800839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.067341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422020+0000","last_change":"2024-04-25T11:19:26.435639+0000","last_active":"2024-04-25T11:19:28.422020+0000","last_peered":"2024-04-25T11:19:28.422020+0000","last_clean":"2024-04-25T11:19:28.422020+0000","last_became_active":"2024-04-25T11:19:26.435051+0000","last_became_peered":"2024-04-25T11:19:26.435051+0000","last_unstale":"2024-04-25T11:19:28.422020+0000","last_undegraded":"2024-04-25T11:19:28.422020+0000","last_fullsized":"2024-04-25T11:19:28.422020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802432+0000","last_change":"2024-04-25T11:19:27.449942+0000","last_active":"2024-04-25T11:19:29.802432+0000","last_peered":"2024-04-25T11:19:29.802432+0000","last_clean":"2024-04-25T11:19:29.802432+0000","last_became_active":"2024-04-25T11:19:27.449419+0000","last_became_peered":"2024-04-25T11:19:27.449419+0000","last_unstale":"2024-04-25T11:19:29.802432+0000","last_undegraded":"2024-04-25T11:19:29.802432+0000","last_fullsized":"2024-04-25T11:19:29.802432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.082294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432820+0000","last_change":"2024-04-25T11:19:26.437548+0000","last_active":"2024-04-25T11:19:28.432820+0000","last_peered":"2024-04-25T11:19:28.432820+0000","last_clean":"2024-04-25T11:19:28.432820+0000","last_became_active":"2024-04-25T11:19:26.436659+0000","last_became_peered":"2024-04-25T11:19:26.436659+0000","last_unstale":"2024-04-25T11:19:28.432820+0000","last_undegraded":"2024-04-25T11:19:28.432820+0000","last_fullsized":"2024-04-25T11:19:28.432820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803625+0000","last_change":"2024-04-25T11:19:27.443841+0000","last_active":"2024-04-25T11:19:29.803625+0000","last_peered":"2024-04-25T11:19:29.803625+0000","last_clean":"2024-04-25T11:19:29.803625+0000","last_became_active":"2024-04-25T11:19:27.443174+0000","last_became_peered":"2024-04-25T11:19:27.443174+0000","last_unstale":"2024-04-25T11:19:29.803625+0000","last_undegraded":"2024-04-25T11:19:29.803625+0000","last_fullsized":"2024-04-25T11:19:29.803625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:52.757499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801465+0000","last_change":"2024-04-25T11:19:26.444467+0000","last_active":"2024-04-25T11:19:29.801465+0000","last_peered":"2024-04-25T11:19:29.801465+0000","last_clean":"2024-04-25T11:19:29.801465+0000","last_became_active":"2024-04-25T11:19:26.444081+0000","last_became_peered":"2024-04-25T11:19:26.444081+0000","last_unstale":"2024-04-25T11:19:29.801465+0000","last_undegraded":"2024-04-25T11:19:29.801465+0000","last_fullsized":"2024-04-25T11:19:29.801465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:10.270275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800928+0000","last_change":"2024-04-25T11:19:27.442198+0000","last_active":"2024-04-25T11:19:29.800928+0000","last_peered":"2024-04-25T11:19:29.800928+0000","last_clean":"2024-04-25T11:19:29.800928+0000","last_became_active":"2024-04-25T11:19:27.441771+0000","last_became_peered":"2024-04-25T11:19:27.441771+0000","last_unstale":"2024-04-25T11:19:29.800928+0000","last_undegraded":"2024-04-25T11:19:29.800928+0000","last_fullsized":"2024-04-25T11:19:29.800928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818060+0000","last_change":"2024-04-25T11:19:26.443383+0000","last_active":"2024-04-25T11:19:28.818060+0000","last_peered":"2024-04-25T11:19:28.818060+0000","last_clean":"2024-04-25T11:19:28.818060+0000","last_became_active":"2024-04-25T11:19:26.442678+0000","last_became_peered":"2024-04-25T11:19:26.442678+0000","last_unstale":"2024-04-25T11:19:28.818060+0000","last_undegraded":"2024-04-25T11:19:28.818060+0000","last_fullsized":"2024-04-25T11:19:28.818060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803299+0000","last_change":"2024-04-25T11:19:27.448069+0000","last_active":"2024-04-25T11:19:29.803299+0000","last_peered":"2024-04-25T11:19:29.803299+0000","last_clean":"2024-04-25T11:19:29.803299+0000","last_became_active":"2024-04-25T11:19:27.447313+0000","last_became_peered":"2024-04-25T11:19:27.447313+0000","last_unstale":"2024-04-25T11:19:29.803299+0000","last_undegraded":"2024-04-25T11:19:29.803299+0000","last_fullsized":"2024-04-25T11:19:29.803299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:03.621864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432496+0000","last_change":"2024-04-25T11:19:26.449988+0000","last_active":"2024-04-25T11:19:28.432496+0000","last_peered":"2024-04-25T11:19:28.432496+0000","last_clean":"2024-04-25T11:19:28.432496+0000","last_became_active":"2024-04-25T11:19:26.445253+0000","last_became_peered":"2024-04-25T11:19:26.445253+0000","last_unstale":"2024-04-25T11:19:28.432496+0000","last_undegraded":"2024-04-25T11:19:28.432496+0000","last_fullsized":"2024-04-25T11:19:28.432496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.432903+0000","last_change":"2024-04-25T11:19:27.423470+0000","last_active":"2024-04-25T11:19:28.432903+0000","last_peered":"2024-04-25T11:19:28.432903+0000","last_clean":"2024-04-25T11:19:28.432903+0000","last_became_active":"2024-04-25T11:19:27.422649+0000","last_became_peered":"2024-04-25T11:19:27.422649+0000","last_unstale":"2024-04-25T11:19:28.432903+0000","last_undegraded":"2024-04-25T11:19:28.432903+0000","last_fullsized":"2024-04-25T11:19:28.432903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:32.622842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.423793+0000","last_change":"2024-04-25T11:19:26.424599+0000","last_active":"2024-04-25T11:19:28.423793+0000","last_peered":"2024-04-25T11:19:28.423793+0000","last_clean":"2024-04-25T11:19:28.423793+0000","last_became_active":"2024-04-25T11:19:26.424110+0000","last_became_peered":"2024-04-25T11:19:26.424110+0000","last_unstale":"2024-04-25T11:19:28.423793+0000","last_undegraded":"2024-04-25T11:19:28.423793+0000","last_fullsized":"2024-04-25T11:19:28.423793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802558+0000","last_change":"2024-04-25T11:19:27.441846+0000","last_active":"2024-04-25T11:19:29.802558+0000","last_peered":"2024-04-25T11:19:29.802558+0000","last_clean":"2024-04-25T11:19:29.802558+0000","last_became_active":"2024-04-25T11:19:27.441349+0000","last_became_peered":"2024-04-25T11:19:27.441349+0000","last_unstale":"2024-04-25T11:19:29.802558+0000","last_undegraded":"2024-04-25T11:19:29.802558+0000","last_fullsized":"2024-04-25T11:19:29.802558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:39.163529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802696+0000","last_change":"2024-04-25T11:19:26.429643+0000","last_active":"2024-04-25T11:19:29.802696+0000","last_peered":"2024-04-25T11:19:29.802696+0000","last_clean":"2024-04-25T11:19:29.802696+0000","last_became_active":"2024-04-25T11:19:26.428760+0000","last_became_peered":"2024-04-25T11:19:26.428760+0000","last_unstale":"2024-04-25T11:19:29.802696+0000","last_undegraded":"2024-04-25T11:19:29.802696+0000","last_fullsized":"2024-04-25T11:19:29.802696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:36.804999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803977+0000","last_change":"2024-04-25T11:19:27.445422+0000","last_active":"2024-04-25T11:19:29.803977+0000","last_peered":"2024-04-25T11:19:29.803977+0000","last_clean":"2024-04-25T11:19:29.803977+0000","last_became_active":"2024-04-25T11:19:27.444978+0000","last_became_peered":"2024-04-25T11:19:27.444978+0000","last_unstale":"2024-04-25T11:19:29.803977+0000","last_undegraded":"2024-04-25T11:19:29.803977+0000","last_fullsized":"2024-04-25T11:19:29.803977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.518039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424380+0000","last_change":"2024-04-25T11:19:26.415971+0000","last_active":"2024-04-25T11:19:28.424380+0000","last_peered":"2024-04-25T11:19:28.424380+0000","last_clean":"2024-04-25T11:19:28.424380+0000","last_became_active":"2024-04-25T11:19:26.415532+0000","last_became_peered":"2024-04-25T11:19:26.415532+0000","last_unstale":"2024-04-25T11:19:28.424380+0000","last_undegraded":"2024-04-25T11:19:28.424380+0000","last_fullsized":"2024-04-25T11:19:28.424380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:52.739386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804821+0000","last_change":"2024-04-25T11:19:27.445828+0000","last_active":"2024-04-25T11:19:29.804821+0000","last_peered":"2024-04-25T11:19:29.804821+0000","last_clean":"2024-04-25T11:19:29.804821+0000","last_became_active":"2024-04-25T11:19:27.445424+0000","last_became_peered":"2024-04-25T11:19:27.445424+0000","last_unstale":"2024-04-25T11:19:29.804821+0000","last_undegraded":"2024-04-25T11:19:29.804821+0000","last_fullsized":"2024-04-25T11:19:29.804821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:41.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422118+0000","last_change":"2024-04-25T11:19:26.442966+0000","last_active":"2024-04-25T11:19:28.422118+0000","last_peered":"2024-04-25T11:19:28.422118+0000","last_clean":"2024-04-25T11:19:28.422118+0000","last_became_active":"2024-04-25T11:19:26.442531+0000","last_became_peered":"2024-04-25T11:19:26.442531+0000","last_unstale":"2024-04-25T11:19:28.422118+0000","last_undegraded":"2024-04-25T11:19:28.422118+0000","last_fullsized":"2024-04-25T11:19:28.422118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818802+0000","last_change":"2024-04-25T11:19:27.454129+0000","last_active":"2024-04-25T11:19:28.818802+0000","last_peered":"2024-04-25T11:19:28.818802+0000","last_clean":"2024-04-25T11:19:28.818802+0000","last_became_active":"2024-04-25T11:19:27.453260+0000","last_became_peered":"2024-04-25T11:19:27.453260+0000","last_unstale":"2024-04-25T11:19:28.818802+0000","last_undegraded":"2024-04-25T11:19:28.818802+0000","last_fullsized":"2024-04-25T11:19:28.818802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.297417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424074+0000","last_change":"2024-04-25T11:19:26.435919+0000","last_active":"2024-04-25T11:19:28.424074+0000","last_peered":"2024-04-25T11:19:28.424074+0000","last_clean":"2024-04-25T11:19:28.424074+0000","last_became_active":"2024-04-25T11:19:26.435172+0000","last_became_peered":"2024-04-25T11:19:26.435172+0000","last_unstale":"2024-04-25T11:19:28.424074+0000","last_undegraded":"2024-04-25T11:19:28.424074+0000","last_fullsized":"2024-04-25T11:19:28.424074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803927+0000","last_change":"2024-04-25T11:19:27.452401+0000","last_active":"2024-04-25T11:19:29.803927+0000","last_peered":"2024-04-25T11:19:29.803927+0000","last_clean":"2024-04-25T11:19:29.803927+0000","last_became_active":"2024-04-25T11:19:27.452007+0000","last_became_peered":"2024-04-25T11:19:27.452007+0000","last_unstale":"2024-04-25T11:19:29.803927+0000","last_undegraded":"2024-04-25T11:19:29.803927+0000","last_fullsized":"2024-04-25T11:19:29.803927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:45.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802096+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:19:29.802096+0000","last_peered":"2024-04-25T11:19:29.802096+0000","last_clean":"2024-04-25T11:19:29.802096+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:19:29.802096+0000","last_undegraded":"2024-04-25T11:19:29.802096+0000","last_fullsized":"2024-04-25T11:19:29.802096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.817551+0000","last_change":"2024-04-25T11:19:26.431881+0000","last_active":"2024-04-25T11:19:28.817551+0000","last_peered":"2024-04-25T11:19:28.817551+0000","last_clean":"2024-04-25T11:19:28.817551+0000","last_became_active":"2024-04-25T11:19:26.431224+0000","last_became_peered":"2024-04-25T11:19:26.431224+0000","last_unstale":"2024-04-25T11:19:28.817551+0000","last_undegraded":"2024-04-25T11:19:28.817551+0000","last_fullsized":"2024-04-25T11:19:28.817551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791934+0000","last_change":"2024-04-25T11:19:27.453756+0000","last_active":"2024-04-25T11:19:29.791934+0000","last_peered":"2024-04-25T11:19:29.791934+0000","last_clean":"2024-04-25T11:19:29.791934+0000","last_became_active":"2024-04-25T11:19:27.453406+0000","last_became_peered":"2024-04-25T11:19:27.453406+0000","last_unstale":"2024-04-25T11:19:29.791934+0000","last_undegraded":"2024-04-25T11:19:29.791934+0000","last_fullsized":"2024-04-25T11:19:29.791934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801787+0000","last_change":"2024-04-25T11:19:26.439003+0000","last_active":"2024-04-25T11:19:29.801787+0000","last_peered":"2024-04-25T11:19:29.801787+0000","last_clean":"2024-04-25T11:19:29.801787+0000","last_became_active":"2024-04-25T11:19:26.437864+0000","last_became_peered":"2024-04-25T11:19:26.437864+0000","last_unstale":"2024-04-25T11:19:29.801787+0000","last_undegraded":"2024-04-25T11:19:29.801787+0000","last_fullsized":"2024-04-25T11:19:29.801787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:03.260366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800291+0000","last_change":"2024-04-25T11:19:27.411133+0000","last_active":"2024-04-25T11:19:29.800291+0000","last_peered":"2024-04-25T11:19:29.800291+0000","last_clean":"2024-04-25T11:19:29.800291+0000","last_became_active":"2024-04-25T11:19:27.410234+0000","last_became_peered":"2024-04-25T11:19:27.410234+0000","last_unstale":"2024-04-25T11:19:29.800291+0000","last_undegraded":"2024-04-25T11:19:29.800291+0000","last_fullsized":"2024-04-25T11:19:29.800291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.618925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.424634+0000","last_change":"2024-04-25T11:19:26.413057+0000","last_active":"2024-04-25T11:19:28.424634+0000","last_peered":"2024-04-25T11:19:28.424634+0000","last_clean":"2024-04-25T11:19:28.424634+0000","last_became_active":"2024-04-25T11:19:26.412586+0000","last_became_peered":"2024-04-25T11:19:26.412586+0000","last_unstale":"2024-04-25T11:19:28.424634+0000","last_undegraded":"2024-04-25T11:19:28.424634+0000","last_fullsized":"2024-04-25T11:19:28.424634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.654002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421086+0000","last_change":"2024-04-25T11:19:27.423812+0000","last_active":"2024-04-25T11:19:28.421086+0000","last_peered":"2024-04-25T11:19:28.421086+0000","last_clean":"2024-04-25T11:19:28.421086+0000","last_became_active":"2024-04-25T11:19:27.419444+0000","last_became_peered":"2024-04-25T11:19:27.419444+0000","last_unstale":"2024-04-25T11:19:28.421086+0000","last_undegraded":"2024-04-25T11:19:28.421086+0000","last_fullsized":"2024-04-25T11:19:28.421086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.687569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818406+0000","last_change":"2024-04-25T11:19:26.433315+0000","last_active":"2024-04-25T11:19:28.818406+0000","last_peered":"2024-04-25T11:19:28.818406+0000","last_clean":"2024-04-25T11:19:28.818406+0000","last_became_active":"2024-04-25T11:19:26.432639+0000","last_became_peered":"2024-04-25T11:19:26.432639+0000","last_unstale":"2024-04-25T11:19:28.818406+0000","last_undegraded":"2024-04-25T11:19:28.818406+0000","last_fullsized":"2024-04-25T11:19:28.818406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:21.523311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801895+0000","last_change":"2024-04-25T11:19:27.415292+0000","last_active":"2024-04-25T11:19:29.801895+0000","last_peered":"2024-04-25T11:19:29.801895+0000","last_clean":"2024-04-25T11:19:29.801895+0000","last_became_active":"2024-04-25T11:19:27.414270+0000","last_became_peered":"2024-04-25T11:19:27.414270+0000","last_unstale":"2024-04-25T11:19:29.801895+0000","last_undegraded":"2024-04-25T11:19:29.801895+0000","last_fullsized":"2024-04-25T11:19:29.801895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.534287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421617+0000","last_change":"2024-04-25T11:19:26.424289+0000","last_active":"2024-04-25T11:19:28.421617+0000","last_peered":"2024-04-25T11:19:28.421617+0000","last_clean":"2024-04-25T11:19:28.421617+0000","last_became_active":"2024-04-25T11:19:26.423826+0000","last_became_peered":"2024-04-25T11:19:26.423826+0000","last_unstale":"2024-04-25T11:19:28.421617+0000","last_undegraded":"2024-04-25T11:19:28.421617+0000","last_fullsized":"2024-04-25T11:19:28.421617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.843624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818628+0000","last_change":"2024-04-25T11:19:27.457615+0000","last_active":"2024-04-25T11:19:28.818628+0000","last_peered":"2024-04-25T11:19:28.818628+0000","last_clean":"2024-04-25T11:19:28.818628+0000","last_became_active":"2024-04-25T11:19:27.447354+0000","last_became_peered":"2024-04-25T11:19:27.447354+0000","last_unstale":"2024-04-25T11:19:28.818628+0000","last_undegraded":"2024-04-25T11:19:28.818628+0000","last_fullsized":"2024-04-25T11:19:28.818628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.518039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.804115+0000","last_change":"2024-04-25T11:19:26.459930+0000","last_active":"2024-04-25T11:19:29.804115+0000","last_peered":"2024-04-25T11:19:29.804115+0000","last_clean":"2024-04-25T11:19:29.804115+0000","last_became_active":"2024-04-25T11:19:26.459292+0000","last_became_peered":"2024-04-25T11:19:26.459292+0000","last_unstale":"2024-04-25T11:19:29.804115+0000","last_undegraded":"2024-04-25T11:19:29.804115+0000","last_fullsized":"2024-04-25T11:19:29.804115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:08.607950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422340+0000","last_change":"2024-04-25T11:19:27.422008+0000","last_active":"2024-04-25T11:19:28.422340+0000","last_peered":"2024-04-25T11:19:28.422340+0000","last_clean":"2024-04-25T11:19:28.422340+0000","last_became_active":"2024-04-25T11:19:27.419929+0000","last_became_peered":"2024-04-25T11:19:27.419929+0000","last_unstale":"2024-04-25T11:19:28.422340+0000","last_undegraded":"2024-04-25T11:19:28.422340+0000","last_fullsized":"2024-04-25T11:19:28.422340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:12.077625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.431242+0000","last_change":"2024-04-25T11:19:26.435682+0000","last_active":"2024-04-25T11:19:28.431242+0000","last_peered":"2024-04-25T11:19:28.431242+0000","last_clean":"2024-04-25T11:19:28.431242+0000","last_became_active":"2024-04-25T11:19:26.435173+0000","last_became_peered":"2024-04-25T11:19:26.435173+0000","last_unstale":"2024-04-25T11:19:28.431242+0000","last_undegraded":"2024-04-25T11:19:28.431242+0000","last_fullsized":"2024-04-25T11:19:28.431242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:49.071318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792892+0000","last_change":"2024-04-25T11:19:27.446933+0000","last_active":"2024-04-25T11:19:29.792892+0000","last_peered":"2024-04-25T11:19:29.792892+0000","last_clean":"2024-04-25T11:19:29.792892+0000","last_became_active":"2024-04-25T11:19:27.446601+0000","last_became_peered":"2024-04-25T11:19:27.446601+0000","last_unstale":"2024-04-25T11:19:29.792892+0000","last_undegraded":"2024-04-25T11:19:29.792892+0000","last_fullsized":"2024-04-25T11:19:29.792892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:39.163529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433109+0000","last_change":"2024-04-25T11:19:26.451159+0000","last_active":"2024-04-25T11:19:28.433109+0000","last_peered":"2024-04-25T11:19:28.433109+0000","last_clean":"2024-04-25T11:19:28.433109+0000","last_became_active":"2024-04-25T11:19:26.450440+0000","last_became_peered":"2024-04-25T11:19:26.450440+0000","last_unstale":"2024-04-25T11:19:28.433109+0000","last_undegraded":"2024-04-25T11:19:28.433109+0000","last_fullsized":"2024-04-25T11:19:28.433109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801203+0000","last_change":"2024-04-25T11:19:27.428326+0000","last_active":"2024-04-25T11:19:29.801203+0000","last_peered":"2024-04-25T11:19:29.801203+0000","last_clean":"2024-04-25T11:19:29.801203+0000","last_became_active":"2024-04-25T11:19:27.427906+0000","last_became_peered":"2024-04-25T11:19:27.427906+0000","last_unstale":"2024-04-25T11:19:29.801203+0000","last_undegraded":"2024-04-25T11:19:29.801203+0000","last_fullsized":"2024-04-25T11:19:29.801203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801345+0000","last_change":"2024-04-25T11:19:26.447443+0000","last_active":"2024-04-25T11:19:29.801345+0000","last_peered":"2024-04-25T11:19:29.801345+0000","last_clean":"2024-04-25T11:19:29.801345+0000","last_became_active":"2024-04-25T11:19:26.439787+0000","last_became_peered":"2024-04-25T11:19:26.439787+0000","last_unstale":"2024-04-25T11:19:29.801345+0000","last_undegraded":"2024-04-25T11:19:29.801345+0000","last_fullsized":"2024-04-25T11:19:29.801345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:49.071318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801449+0000","last_change":"2024-04-25T11:19:27.422934+0000","last_active":"2024-04-25T11:19:29.801449+0000","last_peered":"2024-04-25T11:19:29.801449+0000","last_clean":"2024-04-25T11:19:29.801449+0000","last_became_active":"2024-04-25T11:19:27.421387+0000","last_became_peered":"2024-04-25T11:19:27.421387+0000","last_unstale":"2024-04-25T11:19:29.801449+0000","last_undegraded":"2024-04-25T11:19:29.801449+0000","last_fullsized":"2024-04-25T11:19:29.801449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.963336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.421663+0000","last_change":"2024-04-25T11:19:26.456701+0000","last_active":"2024-04-25T11:19:28.421663+0000","last_peered":"2024-04-25T11:19:28.421663+0000","last_clean":"2024-04-25T11:19:28.421663+0000","last_became_active":"2024-04-25T11:19:26.456252+0000","last_became_peered":"2024-04-25T11:19:26.456252+0000","last_unstale":"2024-04-25T11:19:28.421663+0000","last_undegraded":"2024-04-25T11:19:28.421663+0000","last_fullsized":"2024-04-25T11:19:28.421663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:51.952361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.791806+0000","last_change":"2024-04-25T11:19:27.435245+0000","last_active":"2024-04-25T11:19:29.791806+0000","last_peered":"2024-04-25T11:19:29.791806+0000","last_clean":"2024-04-25T11:19:29.791806+0000","last_became_active":"2024-04-25T11:19:27.434609+0000","last_became_peered":"2024-04-25T11:19:27.434609+0000","last_unstale":"2024-04-25T11:19:29.791806+0000","last_undegraded":"2024-04-25T11:19:29.791806+0000","last_fullsized":"2024-04-25T11:19:29.791806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:58.501421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.422471+0000","last_change":"2024-04-25T11:19:26.423167+0000","last_active":"2024-04-25T11:19:28.422471+0000","last_peered":"2024-04-25T11:19:28.422471+0000","last_clean":"2024-04-25T11:19:28.422471+0000","last_became_active":"2024-04-25T11:19:26.422397+0000","last_became_peered":"2024-04-25T11:19:26.422397+0000","last_unstale":"2024-04-25T11:19:28.422471+0000","last_undegraded":"2024-04-25T11:19:28.422471+0000","last_fullsized":"2024-04-25T11:19:28.422471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.048030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792180+0000","last_change":"2024-04-25T11:19:27.426394+0000","last_active":"2024-04-25T11:19:29.792180+0000","last_peered":"2024-04-25T11:19:29.792180+0000","last_clean":"2024-04-25T11:19:29.792180+0000","last_became_active":"2024-04-25T11:19:27.424504+0000","last_became_peered":"2024-04-25T11:19:27.424504+0000","last_unstale":"2024-04-25T11:19:29.792180+0000","last_undegraded":"2024-04-25T11:19:29.792180+0000","last_fullsized":"2024-04-25T11:19:29.792180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.850268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803567+0000","last_change":"2024-04-25T11:19:26.455784+0000","last_active":"2024-04-25T11:19:29.803567+0000","last_peered":"2024-04-25T11:19:29.803567+0000","last_clean":"2024-04-25T11:19:29.803567+0000","last_became_active":"2024-04-25T11:19:26.455125+0000","last_became_peered":"2024-04-25T11:19:26.455125+0000","last_unstale":"2024-04-25T11:19:29.803567+0000","last_undegraded":"2024-04-25T11:19:29.803567+0000","last_fullsized":"2024-04-25T11:19:29.803567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:40.075026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.433197+0000","last_change":"2024-04-25T11:19:27.455131+0000","last_active":"2024-04-25T11:19:28.433197+0000","last_peered":"2024-04-25T11:19:28.433197+0000","last_clean":"2024-04-25T11:19:28.433197+0000","last_became_active":"2024-04-25T11:19:27.454732+0000","last_became_peered":"2024-04-25T11:19:27.454732+0000","last_unstale":"2024-04-25T11:19:28.433197+0000","last_undegraded":"2024-04-25T11:19:28.433197+0000","last_fullsized":"2024-04-25T11:19:28.433197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818302+0000","last_change":"2024-04-25T11:19:27.450900+0000","last_active":"2024-04-25T11:19:28.818302+0000","last_peered":"2024-04-25T11:19:28.818302+0000","last_clean":"2024-04-25T11:19:28.818302+0000","last_became_active":"2024-04-25T11:19:27.447933+0000","last_became_peered":"2024-04-25T11:19:27.447933+0000","last_unstale":"2024-04-25T11:19:28.818302+0000","last_undegraded":"2024-04-25T11:19:28.818302+0000","last_fullsized":"2024-04-25T11:19:28.818302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:03.621864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.802750+0000","last_change":"2024-04-25T11:19:26.456557+0000","last_active":"2024-04-25T11:19:29.802750+0000","last_peered":"2024-04-25T11:19:29.802750+0000","last_clean":"2024-04-25T11:19:29.802750+0000","last_became_active":"2024-04-25T11:19:26.455740+0000","last_became_peered":"2024-04-25T11:19:26.455740+0000","last_unstale":"2024-04-25T11:19:29.802750+0000","last_undegraded":"2024-04-25T11:19:29.802750+0000","last_fullsized":"2024-04-25T11:19:29.802750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:12.867967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:28.818605+0000","last_change":"2024-04-25T11:19:27.440111+0000","last_active":"2024-04-25T11:19:28.818605+0000","last_peered":"2024-04-25T11:19:28.818605+0000","last_clean":"2024-04-25T11:19:28.818605+0000","last_became_active":"2024-04-25T11:19:27.430503+0000","last_became_peered":"2024-04-25T11:19:27.430503+0000","last_unstale":"2024-04-25T11:19:28.818605+0000","last_undegraded":"2024-04-25T11:19:28.818605+0000","last_fullsized":"2024-04-25T11:19:28.818605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:45.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.803868+0000","last_change":"2024-04-25T11:19:26.457866+0000","last_active":"2024-04-25T11:19:29.803868+0000","last_peered":"2024-04-25T11:19:29.803868+0000","last_clean":"2024-04-25T11:19:29.803868+0000","last_became_active":"2024-04-25T11:19:26.457223+0000","last_became_peered":"2024-04-25T11:19:26.457223+0000","last_unstale":"2024-04-25T11:19:29.803868+0000","last_undegraded":"2024-04-25T11:19:29.803868+0000","last_fullsized":"2024-04-25T11:19:29.803868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.048030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.801205+0000","last_change":"2024-04-25T11:19:27.448305+0000","last_active":"2024-04-25T11:19:29.801205+0000","last_peered":"2024-04-25T11:19:29.801205+0000","last_clean":"2024-04-25T11:19:29.801205+0000","last_became_active":"2024-04-25T11:19:27.447940+0000","last_became_peered":"2024-04-25T11:19:27.447940+0000","last_unstale":"2024-04-25T11:19:29.801205+0000","last_undegraded":"2024-04-25T11:19:29.801205+0000","last_fullsized":"2024-04-25T11:19:29.801205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.589922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792735+0000","last_change":"2024-04-25T11:19:26.425844+0000","last_active":"2024-04-25T11:19:29.792735+0000","last_peered":"2024-04-25T11:19:29.792735+0000","last_clean":"2024-04-25T11:19:29.792735+0000","last_became_active":"2024-04-25T11:19:26.425443+0000","last_became_peered":"2024-04-25T11:19:26.425443+0000","last_unstale":"2024-04-25T11:19:29.792735+0000","last_undegraded":"2024-04-25T11:19:29.792735+0000","last_fullsized":"2024-04-25T11:19:29.792735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.911517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.792425+0000","last_change":"2024-04-25T11:19:27.434918+0000","last_active":"2024-04-25T11:19:29.792425+0000","last_peered":"2024-04-25T11:19:29.792425+0000","last_clean":"2024-04-25T11:19:29.792425+0000","last_became_active":"2024-04-25T11:19:27.434444+0000","last_became_peered":"2024-04-25T11:19:27.434444+0000","last_unstale":"2024-04-25T11:19:29.792425+0000","last_undegraded":"2024-04-25T11:19:29.792425+0000","last_fullsized":"2024-04-25T11:19:29.792425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:26.385075+0000","last_clean_scrub_stamp":"2024-04-25T11:19:26.385075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.398422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:19:29.800979+0000","last_change":"2024-04-25T11:19:26.448774+0000","last_active":"2024-04-25T11:19:29.800979+0000","last_peered":"2024-04-25T11:19:29.800979+0000","last_clean":"2024-04-25T11:19:29.800979+0000","last_became_active":"2024-04-25T11:19:26.448304+0000","last_became_peered":"2024-04-25T11:19:26.448304+0000","last_unstale":"2024-04-25T11:19:29.800979+0000","last_undegraded":"2024-04-25T11:19:29.800979+0000","last_fullsized":"2024-04-25T11:19:29.800979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:25.374100+0000","last_clean_scrub_stamp":"2024-04-25T11:19:25.374100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:46.982993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:19:36.412 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:19:36.412 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:19:36.412 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:19:36.413 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:19:36.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.581+0000 7f35baedb640 1 Processor -- start 2024-04-25T11:19:36.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.581+0000 7f35baedb640 1 -- start start 2024-04-25T11:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35baedb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b415c270 0x7f35b40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35baedb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f35b40cf200 0x7f35b4072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35baedb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f35b40b43f0 0x7f35b4072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35baedb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f35b415fdb0 con 0x7f35b40cf200 2024-04-25T11:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35baedb640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f35b40b49b0 con 0x7f35b40b43f0 2024-04-25T11:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35baedb640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35b40b4c90 con 0x7f35b415c270 2024-04-25T11:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35b3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f35b40cf200 0x7f35b4072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35b3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f35b40cf200 0x7f35b4072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44506/0 (socket says 172.21.15.106:44506) 2024-04-25T11:19:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35b3fff640 1 -- 172.21.15.106:0/2679693212 learned_addr learned my addr 172.21.15.106:0/2679693212 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35b3fff640 1 -- 172.21.15.106:0/2679693212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b415c270 msgr2=0x7f35b40cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35b3fff640 1 --2- 172.21.15.106:0/2679693212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b415c270 0x7f35b40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35b3fff640 1 -- 172.21.15.106:0/2679693212 >> v2:172.21.15.134:3300/0 conn(0x7f35b40b43f0 msgr2=0x7f35b4072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35b3fff640 1 --2- 172.21.15.106:0/2679693212 >> v2:172.21.15.134:3300/0 conn(0x7f35b40b43f0 0x7f35b4072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.585+0000 7f35b3fff640 1 -- 172.21.15.106:0/2679693212 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b40b4f70 con 0x7f35b40cf200 2024-04-25T11:19:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35b3fff640 1 --2- 172.21.15.106:0/2679693212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b40cf200 0x7f35b4072a50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f35b4070e90 tx=0x7f359c01d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279477b86a2eb0bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35b37fe640 1 -- 172.21.15.106:0/2679693212 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0ab610 con 0x7f35b40cf200 2024-04-25T11:19:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35b37fe640 1 -- 172.21.15.106:0/2679693212 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c0b0070 con 0x7f35b40cf200 2024-04-25T11:19:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35b37fe640 1 -- 172.21.15.106:0/2679693212 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0ab9d0 con 0x7f35b40cf200 2024-04-25T11:19:36.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 -- 172.21.15.106:0/2679693212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b40cf200 msgr2=0x7f35b4072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 --2- 172.21.15.106:0/2679693212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b40cf200 0x7f35b4072a50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f35b4070e90 tx=0x7f359c01d650 comp rx=0 tx=0).stop 2024-04-25T11:19:36.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 -- 172.21.15.106:0/2679693212 shutdown_connections 2024-04-25T11:19:36.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 --2- 172.21.15.106:0/2679693212 >> v2:172.21.15.134:3300/0 conn(0x7f35b40b43f0 0x7f35b4072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 --2- 172.21.15.106:0/2679693212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b40cf200 0x7f35b4072a50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 --2- 172.21.15.106:0/2679693212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b415c270 0x7f35b40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 -- 172.21.15.106:0/2679693212 >> 172.21.15.106:0/2679693212 conn(0x7f35b406c7c0 msgr2=0x7f35b406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 -- 172.21.15.106:0/2679693212 shutdown_connections 2024-04-25T11:19:36.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 -- 172.21.15.106:0/2679693212 wait complete. 2024-04-25T11:19:36.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 Processor -- start 2024-04-25T11:19:36.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.589+0000 7f35baedb640 1 -- start start 2024-04-25T11:19:36.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35b40b43f0 0x7f35b4157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b40cf200 0x7f35b415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b415c270 0x7f35b4157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41605d0 con 0x7f35b415c270 2024-04-25T11:19:36.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4160450 con 0x7f35b40b43f0 2024-04-25T11:19:36.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35b4160750 con 0x7f35b40cf200 2024-04-25T11:19:36.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b415c270 0x7f35b4157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b415c270 0x7f35b4157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44510/0 (socket says 172.21.15.106:44510) 2024-04-25T11:19:36.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 -- 172.21.15.106:0/1176533762 learned_addr learned my addr 172.21.15.106:0/1176533762 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:36.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b8c77640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35b40b43f0 0x7f35b4157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b3fff640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b40cf200 0x7f35b415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 -- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b40cf200 msgr2=0x7f35b415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b40cf200 0x7f35b415a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 -- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35b40b43f0 msgr2=0x7f35b4157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35b40b43f0 0x7f35b4157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 -- 172.21.15.106:0/1176533762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b413dcf0 con 0x7f35b415c270 2024-04-25T11:19:36.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b8c77640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35b40b43f0 0x7f35b4157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:36.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b9478640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b415c270 0x7f35b4157180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35a400ba10 tx=0x7f35a400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b17fa640 1 -- 172.21.15.106:0/1176533762 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4019070 con 0x7f35b415c270 2024-04-25T11:19:36.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b17fa640 1 -- 172.21.15.106:0/1176533762 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a400cb40 con 0x7f35b415c270 2024-04-25T11:19:36.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b17fa640 1 -- 172.21.15.106:0/1176533762 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4014440 con 0x7f35b415c270 2024-04-25T11:19:36.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b413dfd0 con 0x7f35b415c270 2024-04-25T11:19:36.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b413e590 con 0x7f35b415c270 2024-04-25T11:19:36.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b40b2b00 con 0x7f35b415c270 2024-04-25T11:19:36.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b17fa640 1 -- 172.21.15.106:0/1176533762 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f35a401c070 con 0x7f35b415c270 2024-04-25T11:19:36.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b17fa640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f359406f430 0x7f3594071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b17fa640 1 -- 172.21.15.106:0/1176533762 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f35a408ef70 con 0x7f35b415c270 2024-04-25T11:19:36.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b8c77640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f359406f430 0x7f3594071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.593+0000 7f35b8c77640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f359406f430 0x7f3594071940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00a870 tx=0x7f35ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.597+0000 7f35b17fa640 1 -- 172.21.15.106:0/1176533762 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f35a405be60 con 0x7f35b415c270 2024-04-25T11:19:36.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.781+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f35b406da30 con 0x7f35b415c270 2024-04-25T11:19:36.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.781+0000 7f35b17fa640 1 -- 172.21.15.106:0/1176533762 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f35a405b760 con 0x7f35b415c270 2024-04-25T11:19:36.781 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:36.781 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:19:36.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f359406f430 msgr2=0x7f3594071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f359406f430 0x7f3594071940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00a870 tx=0x7f35ac008040 comp rx=0 tx=0).stop 2024-04-25T11:19:36.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b415c270 msgr2=0x7f35b4157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b415c270 0x7f35b4157180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35a400ba10 tx=0x7f35a400bed0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 shutdown_connections 2024-04-25T11:19:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f359406f430 0x7f3594071940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b415c270 0x7f35b4157180 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35b40cf200 0x7f35b415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 --2- 172.21.15.106:0/1176533762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35b40b43f0 0x7f35b4157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 >> 172.21.15.106:0/1176533762 conn(0x7f35b406c7c0 msgr2=0x7f35b40b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 shutdown_connections 2024-04-25T11:19:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.785+0000 7f35baedb640 1 -- 172.21.15.106:0/1176533762 wait complete. 2024-04-25T11:19:36.801 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:19:36.801 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:36.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.933+0000 7f4bb84bd640 1 Processor -- start 2024-04-25T11:19:36.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.933+0000 7f4bb84bd640 1 -- start start 2024-04-25T11:19:36.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb84bd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4bb015e0a0 0x7f4bb0073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb84bd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4bb00d0ce0 0x7f4bb00d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb84bd640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb00ce050 con 0x7f4bb00ce850 2024-04-25T11:19:36.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb84bd640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb015e6b0 con 0x7f4bb015e0a0 2024-04-25T11:19:36.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb84bd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb015e990 con 0x7f4bb00d0ce0 2024-04-25T11:19:36.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6259640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4bb015e0a0 0x7f4bb0073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6259640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4bb015e0a0 0x7f4bb0073460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44524/0 (socket says 172.21.15.106:44524) 2024-04-25T11:19:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6259640 1 -- 172.21.15.106:0/905348666 learned_addr learned my addr 172.21.15.106:0/905348666 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6a5a640 1 --2- 172.21.15.106:0/905348666 >> v2:172.21.15.134:3300/0 conn(0x7f4bb00d0ce0 0x7f4bb00d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6259640 1 -- 172.21.15.106:0/905348666 >> v2:172.21.15.134:3300/0 conn(0x7f4bb00d0ce0 msgr2=0x7f4bb00d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6259640 1 --2- 172.21.15.106:0/905348666 >> v2:172.21.15.134:3300/0 conn(0x7f4bb00d0ce0 0x7f4bb00d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6259640 1 -- 172.21.15.106:0/905348666 >> v1:172.21.15.134:6789/0 conn(0x7f4bb00ce850 legacy=0x7f4bb00cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6259640 1 -- 172.21.15.106:0/905348666 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb015ec70 con 0x7f4bb015e0a0 2024-04-25T11:19:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.937+0000 7f4bb6a5a640 1 --2- 172.21.15.106:0/905348666 >> v2:172.21.15.134:3300/0 conn(0x7f4bb00d0ce0 0x7f4bb00d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:36.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb6259640 1 --2- 172.21.15.106:0/905348666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb0073460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4bb006f060 tx=0x7f4bac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22480cf3c774ff59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb5257640 1 -- 172.21.15.106:0/905348666 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0b0070 con 0x7f4bb015e0a0 2024-04-25T11:19:36.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb5257640 1 -- 172.21.15.106:0/905348666 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac0ab600 con 0x7f4bb015e0a0 2024-04-25T11:19:36.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb5257640 1 -- 172.21.15.106:0/905348666 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0ab9d0 con 0x7f4bb015e0a0 2024-04-25T11:19:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 -- 172.21.15.106:0/905348666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 msgr2=0x7f4bb0073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/905348666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb0073460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4bb006f060 tx=0x7f4bac01c4a0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 -- 172.21.15.106:0/905348666 shutdown_connections 2024-04-25T11:19:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/905348666 >> v2:172.21.15.134:3300/0 conn(0x7f4bb00d0ce0 0x7f4bb00d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/905348666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb0073460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 -- 172.21.15.106:0/905348666 >> 172.21.15.106:0/905348666 conn(0x7f4bb006c7c0 msgr2=0x7f4bb006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 -- 172.21.15.106:0/905348666 shutdown_connections 2024-04-25T11:19:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 -- 172.21.15.106:0/905348666 wait complete. 2024-04-25T11:19:36.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 Processor -- start 2024-04-25T11:19:36.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.941+0000 7f4bb84bd640 1 -- start start 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bb00ce850 0x7f4bb0146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb00d0ce0 0x7f4bb0159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb016a870 con 0x7f4bb015e0a0 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb016a6f0 con 0x7f4bb00d0ce0 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb016a9f0 con 0x7f4bb00ce850 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb015a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44526/0 (socket says 172.21.15.106:44526) 2024-04-25T11:19:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 -- 172.21.15.106:0/3184738167 learned_addr learned my addr 172.21.15.106:0/3184738167 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb5a58640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb00d0ce0 0x7f4bb0159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6259640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bb00ce850 0x7f4bb0146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 -- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bb00ce850 msgr2=0x7f4bb0146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bb00ce850 0x7f4bb0146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 -- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb00d0ce0 msgr2=0x7f4bb0159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb00d0ce0 0x7f4bb0159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 -- 172.21.15.106:0/3184738167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb0147c50 con 0x7f4bb015e0a0 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6259640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bb00ce850 0x7f4bb0146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6a5a640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb015a040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800cd50 tx=0x7f4ba800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4ba6ffd640 1 -- 172.21.15.106:0/3184738167 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800f680 con 0x7f4bb015e0a0 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4ba6ffd640 1 -- 172.21.15.106:0/3184738167 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba800f840 con 0x7f4bb015e0a0 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb0147f30 con 0x7f4bb015e0a0 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4ba6ffd640 1 -- 172.21.15.106:0/3184738167 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800fc30 con 0x7f4bb015e0a0 2024-04-25T11:19:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb01484f0 con 0x7f4bb015e0a0 2024-04-25T11:19:36.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb0146e70 con 0x7f4bb015e0a0 2024-04-25T11:19:36.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4ba6ffd640 1 -- 172.21.15.106:0/3184738167 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4ba80213f0 con 0x7f4bb015e0a0 2024-04-25T11:19:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4ba6ffd640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9406f400 0x7f4b94071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4bb6259640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9406f400 0x7f4b94071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.945+0000 7f4ba6ffd640 1 -- 172.21.15.106:0/3184738167 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4ba808ffe0 con 0x7f4bb015e0a0 2024-04-25T11:19:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.949+0000 7f4bb6259640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9406f400 0x7f4b94071910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0ad000 tx=0x7f4bac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:36.953+0000 7f4ba6ffd640 1 -- 172.21.15.106:0/3184738167 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4ba8014030 con 0x7f4bb015e0a0 2024-04-25T11:19:37.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.101+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bb00b2b10 con 0x7f4bb015e0a0 2024-04-25T11:19:37.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.101+0000 7f4ba6ffd640 1 -- 172.21.15.106:0/3184738167 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f4ba805cf60 con 0x7f4bb015e0a0 2024-04-25T11:19:37.101 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:37.102 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:37.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9406f400 msgr2=0x7f4b94071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9406f400 0x7f4b94071910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0ad000 tx=0x7f4bac008040 comp rx=0 tx=0).stop 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 msgr2=0x7f4bb015a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb015a040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800cd50 tx=0x7f4ba800a650 comp rx=0 tx=0).stop 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 shutdown_connections 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9406f400 0x7f4b94071910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bb015e0a0 0x7f4bb015a040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb00d0ce0 0x7f4bb0159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 --2- 172.21.15.106:0/3184738167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bb00ce850 0x7f4bb0146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 >> 172.21.15.106:0/3184738167 conn(0x7f4bb006c7c0 msgr2=0x7f4bb00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 shutdown_connections 2024-04-25T11:19:37.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.105+0000 7f4bb84bd640 1 -- 172.21.15.106:0/3184738167 wait complete. 2024-04-25T11:19:37.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:37.122 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:37.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f689c910640 1 Processor -- start 2024-04-25T11:19:37.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f689c910640 1 -- start start 2024-04-25T11:19:37.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f689c910640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 0x7f689806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f689c910640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68980b68e0 con 0x7f68980d1740 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f689c910640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6898161210 con 0x7f689815d230 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f689c910640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f68980b6290 con 0x7f68980d0cf0 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f6895d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f68980d1740 0x7f68980d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54258/0 (socket says 172.21.15.106:54258) 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f6895d9b640 1 -- 172.21.15.106:0/4231817061 learned_addr learned my addr 172.21.15.106:0/4231817061 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f689659c640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 0x7f689806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.265+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905789360 0 0) 0x7f68980b68e0 con 0x7f68980d1740 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6880003530 con 0x7f68980d1740 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1752341626 0 0) 0x7f6880003530 con 0x7f68980d1740 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 msgr2=0x7f689806ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 0x7f689806ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 >> v1:172.21.15.134:6789/0 conn(0x7f689815d230 legacy=0x7f689815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68980b6570 con 0x7f68980d1740 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f688c0016a0 con 0x7f68980d1740 2024-04-25T11:19:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 >> v1:172.21.15.106:6789/0 conn(0x7f68980d1740 legacy=0x7f68980d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 0x7f68800039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6880003f70 0x7f6880004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6880004d50 0x7f6880005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68980b6290 con 0x7f6880003f70 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f688000bc90 con 0x7f6880004d50 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f688000e0a0 con 0x7f68980d0cf0 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f6895d9b640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6880003f70 0x7f6880004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689659c640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 0x7f68800039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f6895d9b640 1 -- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 msgr2=0x7f68800039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f6895d9b640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 0x7f68800039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f6895d9b640 1 -- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6880004d50 msgr2=0x7f6880005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f6895d9b640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6880004d50 0x7f6880005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f6895d9b640 1 -- 172.21.15.106:0/4231817061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6880003530 con 0x7f6880003f70 2024-04-25T11:19:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689659c640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980d0cf0 0x7f68800039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f6895d9b640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6880003f70 0x7f6880004810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c210 tx=0x7f688c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31e61483293b1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c007d40 con 0x7f6880003f70 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688c0a9070 con 0x7f6880003f70 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689559a640 1 -- 172.21.15.106:0/4231817061 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c005d40 con 0x7f6880003f70 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689c910640 1 -- 172.21.15.106:0/4231817061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6880003f70 msgr2=0x7f6880004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689c910640 1 --2- 172.21.15.106:0/4231817061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6880003f70 0x7f6880004810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c210 tx=0x7f688c01cc60 comp rx=0 tx=0).stop 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f6896d9d640 1 -- 172.21.15.106:0/4231817061 reap_dead start 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689c910640 1 -- 172.21.15.106:0/4231817061 shutdown_connections 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689c910640 1 -- 172.21.15.106:0/4231817061 >> 172.21.15.106:0/4231817061 conn(0x7f689806c9f0 msgr2=0x7f689806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689c910640 1 -- 172.21.15.106:0/4231817061 shutdown_connections 2024-04-25T11:19:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.269+0000 7f689c910640 1 -- 172.21.15.106:0/4231817061 wait complete. 2024-04-25T11:19:37.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 Processor -- start 2024-04-25T11:19:37.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 -- start start 2024-04-25T11:19:37.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6880003f70 0x7f6898157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980b0570 0x7f68981f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f689815eff0 0x7f689815f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f689815cd40 con 0x7f689815eff0 2024-04-25T11:19:37.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f689815cbc0 con 0x7f6880003f70 2024-04-25T11:19:37.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6898073230 con 0x7f68980b0570 2024-04-25T11:19:37.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f689815eff0 0x7f689815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f689815eff0 0x7f689815f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44530/0 (socket says 172.21.15.106:44530) 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 -- 172.21.15.106:0/3632653806 learned_addr learned my addr 172.21.15.106:0/3632653806 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 -- 172.21.15.106:0/3632653806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980b0570 msgr2=0x7f68981f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980b0570 0x7f68981f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 -- 172.21.15.106:0/3632653806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6880003f70 msgr2=0x7f6898157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6880003f70 0x7f6898157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 -- 172.21.15.106:0/3632653806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68980b10e0 con 0x7f689815eff0 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6896d9d640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f689815eff0 0x7f689815f400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68980d1310 tx=0x7f689000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6886ffd640 1 -- 172.21.15.106:0/3632653806 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f689815eff0 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6886ffd640 1 -- 172.21.15.106:0/3632653806 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6890005d70 con 0x7f689815eff0 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689815f940 con 0x7f689815eff0 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f6886ffd640 1 -- 172.21.15.106:0/3632653806 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689000a880 con 0x7f689815eff0 2024-04-25T11:19:37.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.273+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68981f5af0 con 0x7f689815eff0 2024-04-25T11:19:37.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.277+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68981f5860 con 0x7f689815eff0 2024-04-25T11:19:37.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.277+0000 7f6886ffd640 1 -- 172.21.15.106:0/3632653806 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f689001a070 con 0x7f689815eff0 2024-04-25T11:19:37.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.277+0000 7f6886ffd640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f687806f400 0x7f6878071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.277+0000 7f6886ffd640 1 -- 172.21.15.106:0/3632653806 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f689008e560 con 0x7f689815eff0 2024-04-25T11:19:37.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.277+0000 7f689659c640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f687806f400 0x7f6878071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.277+0000 7f689659c640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f687806f400 0x7f6878071910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f68980721f0 tx=0x7f6888008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.281+0000 7f6886ffd640 1 -- 172.21.15.106:0/3632653806 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f689815f940 con 0x7f689815eff0 2024-04-25T11:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.429+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6898050750 con 0x7f689815eff0 2024-04-25T11:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.429+0000 7f6886ffd640 1 -- 172.21.15.106:0/3632653806 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f689005b440 con 0x7f689815eff0 2024-04-25T11:19:37.430 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f687806f400 msgr2=0x7f6878071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f687806f400 0x7f6878071910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f68980721f0 tx=0x7f6888008040 comp rx=0 tx=0).stop 2024-04-25T11:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f689815eff0 msgr2=0x7f689815f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f689815eff0 0x7f689815f400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68980d1310 tx=0x7f689000cb90 comp rx=0 tx=0).stop 2024-04-25T11:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 shutdown_connections 2024-04-25T11:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f687806f400 0x7f6878071910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f689815eff0 0x7f689815f400 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68980b0570 0x7f68981f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 --2- 172.21.15.106:0/3632653806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6880003f70 0x7f6898157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 >> 172.21.15.106:0/3632653806 conn(0x7f689806c9f0 msgr2=0x7f68980b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 shutdown_connections 2024-04-25T11:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.433+0000 7f689c910640 1 -- 172.21.15.106:0/3632653806 wait complete. 2024-04-25T11:19:37.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:37.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42faaf1640 1 Processor -- start 2024-04-25T11:19:37.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42faaf1640 1 -- start start 2024-04-25T11:19:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42faaf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f40d0cf0 0x7f42f406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42faaf1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f42f415d230 0x7f42f415d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42faaf1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f40b68e0 con 0x7f42f40d1740 2024-04-25T11:19:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42faaf1640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f42f4161210 con 0x7f42f415d230 2024-04-25T11:19:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42faaf1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f42f40b6290 con 0x7f42f40d0cf0 2024-04-25T11:19:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f42f40d1740 0x7f42f40d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54260/0 (socket says 172.21.15.106:54260) 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f3fff640 1 -- 172.21.15.106:0/2076113237 learned_addr learned my addr 172.21.15.106:0/2076113237 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f908e640 1 --2- 172.21.15.106:0/2076113237 >> v2:172.21.15.106:3300/0 conn(0x7f42f415d230 0x7f42f415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f888d640 1 --2- 172.21.15.106:0/2076113237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f40d0cf0 0x7f42f406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f37fe640 1 -- 172.21.15.106:0/2076113237 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 941578020 0 0) 0x7f42f40b68e0 con 0x7f42f40d1740 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f37fe640 1 -- 172.21.15.106:0/2076113237 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42dc005740 con 0x7f42f40d1740 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f908e640 1 -- 172.21.15.106:0/2076113237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f40d0cf0 msgr2=0x7f42f406ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f908e640 1 --2- 172.21.15.106:0/2076113237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f40d0cf0 0x7f42f406ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f908e640 1 -- 172.21.15.106:0/2076113237 >> v1:172.21.15.106:6789/0 conn(0x7f42f40d1740 legacy=0x7f42f40d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.605+0000 7f42f908e640 1 -- 172.21.15.106:0/2076113237 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f40b6570 con 0x7f42f415d230 2024-04-25T11:19:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42f908e640 1 --2- 172.21.15.106:0/2076113237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f415d230 0x7f42f415d640 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009d90 tx=0x7f42ec01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f4aa2c75d0aecf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42f37fe640 1 -- 172.21.15.106:0/2076113237 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0ac070 con 0x7f42f415d230 2024-04-25T11:19:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42f37fe640 1 -- 172.21.15.106:0/2076113237 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ec0a6620 con 0x7f42f415d230 2024-04-25T11:19:37.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42f37fe640 1 -- 172.21.15.106:0/2076113237 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0a69d0 con 0x7f42f415d230 2024-04-25T11:19:37.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42faaf1640 1 -- 172.21.15.106:0/2076113237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f415d230 msgr2=0x7f42f415d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42faaf1640 1 --2- 172.21.15.106:0/2076113237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f415d230 0x7f42f415d640 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009d90 tx=0x7f42ec01c850 comp rx=0 tx=0).stop 2024-04-25T11:19:37.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42faaf1640 1 -- 172.21.15.106:0/2076113237 shutdown_connections 2024-04-25T11:19:37.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42faaf1640 1 --2- 172.21.15.106:0/2076113237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f415d230 0x7f42f415d640 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42faaf1640 1 --2- 172.21.15.106:0/2076113237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f40d0cf0 0x7f42f406ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42faaf1640 1 -- 172.21.15.106:0/2076113237 >> 172.21.15.106:0/2076113237 conn(0x7f42f406c9f0 msgr2=0x7f42f406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:37.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42faaf1640 1 -- 172.21.15.106:0/2076113237 shutdown_connections 2024-04-25T11:19:37.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.609+0000 7f42faaf1640 1 -- 172.21.15.106:0/2076113237 wait complete. 2024-04-25T11:19:37.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 Processor -- start 2024-04-25T11:19:37.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 -- start start 2024-04-25T11:19:37.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f40d0cf0 0x7f42f414b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f40d1740 0x7f42f4154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f415d230 0x7f42f415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42f4073440 con 0x7f42f40d0cf0 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42f415cdd0 con 0x7f42f40d1740 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f42f40735c0 con 0x7f42f415d230 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f40d0cf0 0x7f42f414b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f40d0cf0 0x7f42f414b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44548/0 (socket says 172.21.15.106:44548) 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 -- 172.21.15.106:0/3963283591 learned_addr learned my addr 172.21.15.106:0/3963283591 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 -- 172.21.15.106:0/3963283591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f415d230 msgr2=0x7f42f415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f415d230 0x7f42f415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f3fff640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f40d1740 0x7f42f4154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 -- 172.21.15.106:0/3963283591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f40d1740 msgr2=0x7f42f4154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f40d1740 0x7f42f4154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 -- 172.21.15.106:0/3963283591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f4155830 con 0x7f42f40d0cf0 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f888d640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f40d0cf0 0x7f42f414b0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f42e400d5f0 tx=0x7f42e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f17fa640 1 -- 172.21.15.106:0/3963283591 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4004240 con 0x7f42f40d0cf0 2024-04-25T11:19:37.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f17fa640 1 -- 172.21.15.106:0/3963283591 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e40043e0 con 0x7f42f40d0cf0 2024-04-25T11:19:37.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f4155b10 con 0x7f42f40d0cf0 2024-04-25T11:19:37.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f17fa640 1 -- 172.21.15.106:0/3963283591 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4003840 con 0x7f42f40d0cf0 2024-04-25T11:19:37.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f4157880 con 0x7f42f40d0cf0 2024-04-25T11:19:37.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f4050750 con 0x7f42f40d0cf0 2024-04-25T11:19:37.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.613+0000 7f42f17fa640 1 -- 172.21.15.106:0/3963283591 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f42e40039e0 con 0x7f42f40d0cf0 2024-04-25T11:19:37.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.617+0000 7f42f17fa640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f42e006f430 0x7f42e0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.617+0000 7f42f17fa640 1 -- 172.21.15.106:0/3963283591 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f42e408f4e0 con 0x7f42f40d0cf0 2024-04-25T11:19:37.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.617+0000 7f42f3fff640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f42e006f430 0x7f42e0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.617+0000 7f42f3fff640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f42e006f430 0x7f42e0071940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e8007fd0 tx=0x7f42e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.621+0000 7f42f17fa640 1 -- 172.21.15.106:0/3963283591 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f42e405c460 con 0x7f42f40d0cf0 2024-04-25T11:19:37.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42f40b11c0 con 0x7f42f40d0cf0 2024-04-25T11:19:37.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42f17fa640 1 -- 172.21.15.106:0/3963283591 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f42e405bd60 con 0x7f42f40d0cf0 2024-04-25T11:19:37.768 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:19:37.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f42e006f430 msgr2=0x7f42e0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42faaf1640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f42e006f430 0x7f42e0071940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e8007fd0 tx=0x7f42e8009040 comp rx=0 tx=0).stop 2024-04-25T11:19:37.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f40d0cf0 msgr2=0x7f42f414b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42faaf1640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f40d0cf0 0x7f42f414b0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f42e400d5f0 tx=0x7f42e400dab0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 shutdown_connections 2024-04-25T11:19:37.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42faaf1640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f415d230 0x7f42f415acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.769+0000 7f42faaf1640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f40d1740 0x7f42f4154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.773+0000 7f42faaf1640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f42e006f430 0x7f42e0071940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.773+0000 7f42faaf1640 1 --2- 172.21.15.106:0/3963283591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f40d0cf0 0x7f42f414b0c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.773+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 >> 172.21.15.106:0/3963283591 conn(0x7f42f406c9f0 msgr2=0x7f42f415e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:37.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.773+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 shutdown_connections 2024-04-25T11:19:37.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.773+0000 7f42faaf1640 1 -- 172.21.15.106:0/3963283591 wait complete. 2024-04-25T11:19:37.788 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:37.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa804468640 1 Processor -- start 2024-04-25T11:19:37.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa804468640 1 -- start start 2024-04-25T11:19:37.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa804468640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa7fc0d0e40 0x7fa7fc161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa804468640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7fc0d5b40 0x7fa7fc0d5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa804468640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc161560 con 0x7fa7fc0d4440 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa804468640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc0d6490 con 0x7fa7fc0d5b40 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa804468640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc0d1400 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa801a03640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7fc0d5b40 0x7fa7fc0d5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa801a03640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7fc0d5b40 0x7fa7fc0d5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44556/0 (socket says 172.21.15.106:44556) 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa801a03640 1 -- 172.21.15.106:0/1421238614 learned_addr learned my addr 172.21.15.106:0/1421238614 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa802204640 1 --2- 172.21.15.106:0/1421238614 >> v2:172.21.15.134:3300/0 conn(0x7fa7fc0d0e40 0x7fa7fc161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa801a03640 1 -- 172.21.15.106:0/1421238614 >> v2:172.21.15.134:3300/0 conn(0x7fa7fc0d0e40 msgr2=0x7fa7fc161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa801a03640 1 --2- 172.21.15.106:0/1421238614 >> v2:172.21.15.134:3300/0 conn(0x7fa7fc0d0e40 0x7fa7fc161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa801a03640 1 -- 172.21.15.106:0/1421238614 >> v1:172.21.15.134:6789/0 conn(0x7fa7fc0d4440 legacy=0x7fa7fc0d4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.929+0000 7fa801a03640 1 -- 172.21.15.106:0/1421238614 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc0d2300 con 0x7fa7fc0d5b40 2024-04-25T11:19:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa801a03640 1 --2- 172.21.15.106:0/1421238614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d5b40 0x7fa7fc0d5f50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0d3310 tx=0x7fa7ec01c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96422b60cb5592b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa801202640 1 -- 172.21.15.106:0/1421238614 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0b0070 con 0x7fa7fc0d5b40 2024-04-25T11:19:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa801202640 1 -- 172.21.15.106:0/1421238614 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ec0ab600 con 0x7fa7fc0d5b40 2024-04-25T11:19:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa801202640 1 -- 172.21.15.106:0/1421238614 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0ab9d0 con 0x7fa7fc0d5b40 2024-04-25T11:19:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa804468640 1 -- 172.21.15.106:0/1421238614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d5b40 msgr2=0x7fa7fc0d5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa804468640 1 --2- 172.21.15.106:0/1421238614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d5b40 0x7fa7fc0d5f50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0d3310 tx=0x7fa7ec01c270 comp rx=0 tx=0).stop 2024-04-25T11:19:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa804468640 1 -- 172.21.15.106:0/1421238614 shutdown_connections 2024-04-25T11:19:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa804468640 1 --2- 172.21.15.106:0/1421238614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d5b40 0x7fa7fc0d5f50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa804468640 1 --2- 172.21.15.106:0/1421238614 >> v2:172.21.15.134:3300/0 conn(0x7fa7fc0d0e40 0x7fa7fc161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa804468640 1 -- 172.21.15.106:0/1421238614 >> 172.21.15.106:0/1421238614 conn(0x7fa7fc073ef0 msgr2=0x7fa7fc074300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa804468640 1 -- 172.21.15.106:0/1421238614 shutdown_connections 2024-04-25T11:19:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.933+0000 7fa804468640 1 -- 172.21.15.106:0/1421238614 wait complete. 2024-04-25T11:19:37.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 Processor -- start 2024-04-25T11:19:37.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 -- start start 2024-04-25T11:19:37.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d0e40 0x7fa7fc0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d4440 0x7fa7fc0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0d5b40 0x7fa7fc154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc16d8c0 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc16d740 con 0x7fa7fc0d4440 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc16da40 con 0x7fa7fc0d5b40 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d0e40 0x7fa7fc0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d0e40 0x7fa7fc0cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44568/0 (socket says 172.21.15.106:44568) 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 -- 172.21.15.106:0/577711663 learned_addr learned my addr 172.21.15.106:0/577711663 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 -- 172.21.15.106:0/577711663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0d5b40 msgr2=0x7fa7fc154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0d5b40 0x7fa7fc154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 -- 172.21.15.106:0/577711663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d4440 msgr2=0x7fa7fc0cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d4440 0x7fa7fc0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 -- 172.21.15.106:0/577711663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc0cce20 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa802204640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d0e40 0x7fa7fc0cf270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800d5d0 tx=0x7fa7f800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa7f2ffd640 1 -- 172.21.15.106:0/577711663 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8004240 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa7f2ffd640 1 -- 172.21.15.106:0/577711663 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f80043e0 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa7f2ffd640 1 -- 172.21.15.106:0/577711663 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8003840 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc0cd2a0 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc0cea70 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.937+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc050750 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.941+0000 7fa7f2ffd640 1 -- 172.21.15.106:0/577711663 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa7f80039e0 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.941+0000 7fa7f2ffd640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa7e006f430 0x7fa7e0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:37.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.941+0000 7fa7f2ffd640 1 -- 172.21.15.106:0/577711663 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa7f808f550 con 0x7fa7fc0d0e40 2024-04-25T11:19:37.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.941+0000 7fa801a03640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa7e006f430 0x7fa7e0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:37.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.941+0000 7fa801a03640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa7e006f430 0x7fa7e0071940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ad000 tx=0x7fa7ec004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:37.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:37.945+0000 7fa7f2ffd640 1 -- 172.21.15.106:0/577711663 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa7f805c4d0 con 0x7fa7fc0d0e40 2024-04-25T11:19:38.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.089+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa7fc0d3f60 con 0x7fa7fc0d0e40 2024-04-25T11:19:38.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa7f2ffd640 1 -- 172.21.15.106:0/577711663 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa7f805bdd0 con 0x7fa7fc0d0e40 2024-04-25T11:19:38.091 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:19:38.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa7e006f430 msgr2=0x7fa7e0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa7e006f430 0x7fa7e0071940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ad000 tx=0x7fa7ec004740 comp rx=0 tx=0).stop 2024-04-25T11:19:38.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d0e40 msgr2=0x7fa7fc0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d0e40 0x7fa7fc0cf270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800d5d0 tx=0x7fa7f800da90 comp rx=0 tx=0).stop 2024-04-25T11:19:38.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 shutdown_connections 2024-04-25T11:19:38.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0d5b40 0x7fa7fc154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d4440 0x7fa7fc0cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa7e006f430 0x7fa7e0071940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 --2- 172.21.15.106:0/577711663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7fc0d0e40 0x7fa7fc0cf270 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800d5d0 tx=0x7fa7f800da90 comp rx=0 tx=0).stop 2024-04-25T11:19:38.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 >> 172.21.15.106:0/577711663 conn(0x7fa7fc073ef0 msgr2=0x7fa7fc161be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 shutdown_connections 2024-04-25T11:19:38.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.093+0000 7fa804468640 1 -- 172.21.15.106:0/577711663 wait complete. 2024-04-25T11:19:38.111 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4c3d7640 1 Processor -- start 2024-04-25T11:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4c3d7640 1 -- start start 2024-04-25T11:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4c3d7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcc440ba970 0x7fcc440d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4c3d7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcc440d2160 0x7fcc440d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4c3d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440b5120 0x7fcc440b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4c3d7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcc440d1100 con 0x7fcc440d2160 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4c3d7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcc440d2ab0 con 0x7fcc440ba970 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4c3d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcc440d0530 con 0x7fcc440b5120 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc49972640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcc440d2160 0x7fcc440d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc49972640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcc440d2160 0x7fcc440d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44582/0 (socket says 172.21.15.106:44582) 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc49972640 1 -- 172.21.15.106:0/1050463632 learned_addr learned my addr 172.21.15.106:0/1050463632 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc49972640 1 -- 172.21.15.106:0/1050463632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440b5120 msgr2=0x7fcc440b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4a173640 1 --2- 172.21.15.106:0/1050463632 >> v2:172.21.15.134:3300/0 conn(0x7fcc440ba970 0x7fcc440d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc4a974640 1 --2- 172.21.15.106:0/1050463632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440b5120 0x7fcc440b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc49972640 1 --2- 172.21.15.106:0/1050463632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440b5120 0x7fcc440b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc49972640 1 -- 172.21.15.106:0/1050463632 >> v2:172.21.15.134:3300/0 conn(0x7fcc440ba970 msgr2=0x7fcc440d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc49972640 1 --2- 172.21.15.106:0/1050463632 >> v2:172.21.15.134:3300/0 conn(0x7fcc440ba970 0x7fcc440d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.245+0000 7fcc49972640 1 -- 172.21.15.106:0/1050463632 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc440d07d0 con 0x7fcc440d2160 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc49972640 1 --2- 172.21.15.106:0/1050463632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440d2160 0x7fcc440d2570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc4408e700 tx=0x7fcc3c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2d7b6d554f67cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc49171640 1 -- 172.21.15.106:0/1050463632 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0a56e0 con 0x7fcc440d2160 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc49171640 1 -- 172.21.15.106:0/1050463632 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3c0a5880 con 0x7fcc440d2160 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc49171640 1 -- 172.21.15.106:0/1050463632 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0a5c50 con 0x7fcc440d2160 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/1050463632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440d2160 msgr2=0x7fcc440d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/1050463632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440d2160 0x7fcc440d2570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc4408e700 tx=0x7fcc3c01ca50 comp rx=0 tx=0).stop 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/1050463632 shutdown_connections 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/1050463632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440b5120 0x7fcc440b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/1050463632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440d2160 0x7fcc440d2570 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/1050463632 >> v2:172.21.15.134:3300/0 conn(0x7fcc440ba970 0x7fcc440d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/1050463632 >> 172.21.15.106:0/1050463632 conn(0x7fcc440d57e0 msgr2=0x7fcc440d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/1050463632 shutdown_connections 2024-04-25T11:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.249+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/1050463632 wait complete. 2024-04-25T11:19:38.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 Processor -- start 2024-04-25T11:19:38.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 -- start start 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440b5120 0x7fcc440cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440ba970 0x7fcc44163450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440d2160 0x7fcc441581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcc440b4da0 con 0x7fcc440ba970 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441605b0 con 0x7fcc440b5120 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcc440b4f20 con 0x7fcc440d2160 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440ba970 0x7fcc44163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440ba970 0x7fcc44163450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44584/0 (socket says 172.21.15.106:44584) 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 -- 172.21.15.106:0/3961482091 learned_addr learned my addr 172.21.15.106:0/3961482091 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4a173640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440b5120 0x7fcc440cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 -- 172.21.15.106:0/3961482091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440d2160 msgr2=0x7fcc441581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440d2160 0x7fcc441581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 -- 172.21.15.106:0/3961482091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440b5120 msgr2=0x7fcc440cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440b5120 0x7fcc440cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 -- 172.21.15.106:0/3961482091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc44161c60 con 0x7fcc440ba970 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4a173640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440b5120 0x7fcc440cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc49972640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440ba970 0x7fcc44163450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009040 tx=0x7fcc3c0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc3affd640 1 -- 172.21.15.106:0/3961482091 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0b8440 con 0x7fcc440ba970 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc3affd640 1 -- 172.21.15.106:0/3961482091 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3c0b8600 con 0x7fcc440ba970 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc3affd640 1 -- 172.21.15.106:0/3961482091 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0b89f0 con 0x7fcc440ba970 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc440cab60 con 0x7fcc440ba970 2024-04-25T11:19:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc44158a30 con 0x7fcc440ba970 2024-04-25T11:19:38.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.253+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc440d2ed0 con 0x7fcc440ba970 2024-04-25T11:19:38.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.257+0000 7fcc3affd640 1 -- 172.21.15.106:0/3961482091 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcc3c0c1020 con 0x7fcc440ba970 2024-04-25T11:19:38.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.257+0000 7fcc3affd640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcc2806f400 0x7fcc28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.257+0000 7fcc3affd640 1 -- 172.21.15.106:0/3961482091 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc3c12a230 con 0x7fcc440ba970 2024-04-25T11:19:38.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.257+0000 7fcc4a173640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcc2806f400 0x7fcc28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.257+0000 7fcc4a173640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcc2806f400 0x7fcc28071910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcc440b8320 tx=0x7fcc34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.261+0000 7fcc3affd640 1 -- 172.21.15.106:0/3961482091 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcc3c0c8020 con 0x7fcc440ba970 2024-04-25T11:19:38.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.449+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc4415eb30 con 0x7fcc440ba970 2024-04-25T11:19:38.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.449+0000 7fcc3affd640 1 -- 172.21.15.106:0/3961482091 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fcc3c016640 con 0x7fcc440ba970 2024-04-25T11:19:38.449 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:38.449 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:26.806158+0000","modified":"2024-04-25T11:19:29.823480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:19:38.450 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T11:19:38.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcc2806f400 msgr2=0x7fcc28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcc2806f400 0x7fcc28071910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcc440b8320 tx=0x7fcc34008040 comp rx=0 tx=0).stop 2024-04-25T11:19:38.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440ba970 msgr2=0x7fcc44163450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440ba970 0x7fcc44163450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009040 tx=0x7fcc3c0197f0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 shutdown_connections 2024-04-25T11:19:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440d2160 0x7fcc441581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcc2806f400 0x7fcc28071910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc440ba970 0x7fcc44163450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 --2- 172.21.15.106:0/3961482091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440b5120 0x7fcc440cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 >> 172.21.15.106:0/3961482091 conn(0x7fcc440d57e0 msgr2=0x7fcc4406de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 shutdown_connections 2024-04-25T11:19:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.453+0000 7fcc4c3d7640 1 -- 172.21.15.106:0/3961482091 wait complete. 2024-04-25T11:19:38.469 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:38.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c2d23640 1 Processor -- start 2024-04-25T11:19:38.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c2d23640 1 -- start start 2024-04-25T11:19:38.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c2d23640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4bc1498a0 0x7fd4bc070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c2d23640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4bc06c840 con 0x7fd4bc06d600 2024-04-25T11:19:38.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c2d23640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4bc06cb20 con 0x7fd4bc06da10 2024-04-25T11:19:38.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c2d23640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd4bc071200 con 0x7fd4bc1498a0 2024-04-25T11:19:38.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c0abf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4bc1498a0 0x7fd4bc070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4bbfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd4bc06d600 0x7fd4bc0aee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54262/0 (socket says 172.21.15.106:54262) 2024-04-25T11:19:38.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4bbfff640 1 -- 172.21.15.106:0/1393441853 learned_addr learned my addr 172.21.15.106:0/1393441853 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:38.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c0abf640 1 -- 172.21.15.106:0/1393441853 >> v1:172.21.15.134:6789/0 conn(0x7fd4bc06da10 legacy=0x7fd4bc0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c0abf640 1 -- 172.21.15.106:0/1393441853 >> v1:172.21.15.106:6789/0 conn(0x7fd4bc06d600 legacy=0x7fd4bc0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.609+0000 7fd4c0abf640 1 -- 172.21.15.106:0/1393441853 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc06ce00 con 0x7fd4bc1498a0 2024-04-25T11:19:38.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4c0abf640 1 --2- 172.21.15.106:0/1393441853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc1498a0 0x7fd4bc070730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00097c0 tx=0x7fd4b001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a762b6520c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4bb7fe640 1 -- 172.21.15.106:0/1393441853 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00b5070 con 0x7fd4bc1498a0 2024-04-25T11:19:38.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4bb7fe640 1 -- 172.21.15.106:0/1393441853 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b00a8990 con 0x7fd4bc1498a0 2024-04-25T11:19:38.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4bb7fe640 1 -- 172.21.15.106:0/1393441853 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00a8d60 con 0x7fd4bc1498a0 2024-04-25T11:19:38.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4c2d23640 1 -- 172.21.15.106:0/1393441853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc1498a0 msgr2=0x7fd4bc070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4c2d23640 1 --2- 172.21.15.106:0/1393441853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc1498a0 0x7fd4bc070730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00097c0 tx=0x7fd4b001ec90 comp rx=0 tx=0).stop 2024-04-25T11:19:38.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4c2d23640 1 -- 172.21.15.106:0/1393441853 shutdown_connections 2024-04-25T11:19:38.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4c2d23640 1 --2- 172.21.15.106:0/1393441853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc1498a0 0x7fd4bc070730 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4c2d23640 1 -- 172.21.15.106:0/1393441853 >> 172.21.15.106:0/1393441853 conn(0x7fd4bc0d1c00 msgr2=0x7fd4bc0d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4c2d23640 1 -- 172.21.15.106:0/1393441853 shutdown_connections 2024-04-25T11:19:38.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.613+0000 7fd4c2d23640 1 -- 172.21.15.106:0/1393441853 wait complete. 2024-04-25T11:19:38.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 Processor -- start 2024-04-25T11:19:38.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 -- start start 2024-04-25T11:19:38.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc06d600 0x7fd4bc1419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4bc06da10 0x7fd4bc0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4bc1498a0 0x7fd4bc0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc06d170 con 0x7fd4bc06d600 2024-04-25T11:19:38.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc06c5f0 con 0x7fd4bc06da10 2024-04-25T11:19:38.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd4bc0af590 con 0x7fd4bc1498a0 2024-04-25T11:19:38.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc06d600 0x7fd4bc1419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc06d600 0x7fd4bc1419d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44592/0 (socket says 172.21.15.106:44592) 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 -- 172.21.15.106:0/3784890959 learned_addr learned my addr 172.21.15.106:0/3784890959 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 -- 172.21.15.106:0/3784890959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4bc1498a0 msgr2=0x7fd4bc0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c12c0640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4bc1498a0 0x7fd4bc0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4bc1498a0 0x7fd4bc0cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 -- 172.21.15.106:0/3784890959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4bc06da10 msgr2=0x7fd4bc0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4bc06da10 0x7fd4bc0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 -- 172.21.15.106:0/3784890959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc161da0 con 0x7fd4bc06d600 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c0abf640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc06d600 0x7fd4bc1419d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00b3040 tx=0x7fd4b001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4b97fa640 1 -- 172.21.15.106:0/3784890959 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00bf070 con 0x7fd4bc06d600 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4b97fa640 1 -- 172.21.15.106:0/3784890959 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b00b08a0 con 0x7fd4bc06d600 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4bc15d590 con 0x7fd4bc06d600 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4b97fa640 1 -- 172.21.15.106:0/3784890959 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00b5040 con 0x7fd4bc06d600 2024-04-25T11:19:38.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4bc15db50 con 0x7fd4bc06d600 2024-04-25T11:19:38.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.617+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4bc1446d0 con 0x7fd4bc06d600 2024-04-25T11:19:38.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.621+0000 7fd4b97fa640 1 -- 172.21.15.106:0/3784890959 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4b00c2070 con 0x7fd4bc06d600 2024-04-25T11:19:38.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.621+0000 7fd4b97fa640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4a806f430 0x7fd4a8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.621+0000 7fd4b97fa640 1 -- 172.21.15.106:0/3784890959 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd4b012bab0 con 0x7fd4bc06d600 2024-04-25T11:19:38.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.621+0000 7fd4bbfff640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4a806f430 0x7fd4a8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.621+0000 7fd4bbfff640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4a806f430 0x7fd4a8071940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc075ec0 tx=0x7fd4ac007a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.625+0000 7fd4b97fa640 1 -- 172.21.15.106:0/3784890959 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4b00187f0 con 0x7fd4bc06d600 2024-04-25T11:19:38.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.785+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4bc075c10 con 0x7fd4bc06d600 2024-04-25T11:19:38.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.785+0000 7fd4b97fa640 1 -- 172.21.15.106:0/3784890959 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7fd4b00f8a30 con 0x7fd4bc06d600 2024-04-25T11:19:38.785 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:38.785 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4a806f430 msgr2=0x7fd4a8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4a806f430 0x7fd4a8071940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc075ec0 tx=0x7fd4ac007a50 comp rx=0 tx=0).stop 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc06d600 msgr2=0x7fd4bc1419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc06d600 0x7fd4bc1419d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00b3040 tx=0x7fd4b001ba30 comp rx=0 tx=0).stop 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 shutdown_connections 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4bc1498a0 0x7fd4bc0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4bc06da10 0x7fd4bc0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd4a806f430 0x7fd4a8071940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 --2- 172.21.15.106:0/3784890959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4bc06d600 0x7fd4bc1419d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 >> 172.21.15.106:0/3784890959 conn(0x7fd4bc0d1c00 msgr2=0x7fd4bc1626a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 shutdown_connections 2024-04-25T11:19:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.789+0000 7fd4c2d23640 1 -- 172.21.15.106:0/3784890959 wait complete. 2024-04-25T11:19:38.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:38.806 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:38.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40eddb7640 1 Processor -- start 2024-04-25T11:19:38.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40eddb7640 1 -- start start 2024-04-25T11:19:38.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40eddb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e80b5730 0x7f40e80b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40eddb7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f40e815fdc0 0x7f40e81601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40eddb7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e80b6080 con 0x7f40e8160710 2024-04-25T11:19:38.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40eddb7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f40e80af390 con 0x7f40e815fdc0 2024-04-25T11:19:38.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40eddb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f40e8160c00 con 0x7f40e80b5730 2024-04-25T11:19:38.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40e6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f40e815fdc0 0x7f40e81601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40e7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f40e8160710 0x7f40e80afb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54270/0 (socket says 172.21.15.106:54270) 2024-04-25T11:19:38.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40e7fff640 1 -- 172.21.15.106:0/3571195717 learned_addr learned my addr 172.21.15.106:0/3571195717 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:38.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40e77fe640 1 --2- 172.21.15.106:0/3571195717 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e80b5730 0x7f40e80b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40e6ffd640 1 -- 172.21.15.106:0/3571195717 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e80b5730 msgr2=0x7f40e80b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:38.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40e6ffd640 1 --2- 172.21.15.106:0/3571195717 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e80b5730 0x7f40e80b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40e6ffd640 1 -- 172.21.15.106:0/3571195717 >> v1:172.21.15.106:6789/0 conn(0x7f40e8160710 legacy=0x7f40e80afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.945+0000 7f40e6ffd640 1 -- 172.21.15.106:0/3571195717 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e8160ee0 con 0x7f40e815fdc0 2024-04-25T11:19:38.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40e6ffd640 1 --2- 172.21.15.106:0/3571195717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e815fdc0 0x7f40e81601d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f40d00097c0 tx=0x7f40d001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefbf67923f9434f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40e67fc640 1 -- 172.21.15.106:0/3571195717 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d00a7070 con 0x7f40e815fdc0 2024-04-25T11:19:38.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40e67fc640 1 -- 172.21.15.106:0/3571195717 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d00b0070 con 0x7f40e815fdc0 2024-04-25T11:19:38.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40e67fc640 1 -- 172.21.15.106:0/3571195717 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d00ab890 con 0x7f40e815fdc0 2024-04-25T11:19:38.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 -- 172.21.15.106:0/3571195717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e815fdc0 msgr2=0x7f40e81601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3571195717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e815fdc0 0x7f40e81601d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f40d00097c0 tx=0x7f40d001c470 comp rx=0 tx=0).stop 2024-04-25T11:19:38.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 -- 172.21.15.106:0/3571195717 shutdown_connections 2024-04-25T11:19:38.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3571195717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e815fdc0 0x7f40e81601d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3571195717 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e80b5730 0x7f40e80b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 -- 172.21.15.106:0/3571195717 >> 172.21.15.106:0/3571195717 conn(0x7f40e809ba30 msgr2=0x7f40e80b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 -- 172.21.15.106:0/3571195717 shutdown_connections 2024-04-25T11:19:38.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 -- 172.21.15.106:0/3571195717 wait complete. 2024-04-25T11:19:38.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 Processor -- start 2024-04-25T11:19:38.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.949+0000 7f40eddb7640 1 -- start start 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e80b5730 0x7f40e81fd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e815fdc0 0x7f40e814d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e8160710 0x7f40e8157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f40e80b0e40 con 0x7f40e80b5730 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f40e80b0cc0 con 0x7f40e815fdc0 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f40e816c980 con 0x7f40e8160710 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e80b5730 0x7f40e81fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e80b5730 0x7f40e81fd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44618/0 (socket says 172.21.15.106:44618) 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 -- 172.21.15.106:0/3837195056 learned_addr learned my addr 172.21.15.106:0/3837195056 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e6ffd640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e815fdc0 0x7f40e814d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e7fff640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e8160710 0x7f40e8157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 -- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e8160710 msgr2=0x7f40e8157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e8160710 0x7f40e8157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 -- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e815fdc0 msgr2=0x7f40e814d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e815fdc0 0x7f40e814d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 -- 172.21.15.106:0/3837195056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e81fd6d0 con 0x7f40e80b5730 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e6ffd640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e815fdc0 0x7f40e814d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e77fe640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e80b5730 0x7f40e81fd190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40d8007e60 tx=0x7f40d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40dffff640 1 -- 172.21.15.106:0/3837195056 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d801a070 con 0x7f40e80b5730 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40dffff640 1 -- 172.21.15.106:0/3837195056 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d8003ba0 con 0x7f40e80b5730 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e8150f10 con 0x7f40e80b5730 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40dffff640 1 -- 172.21.15.106:0/3837195056 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8004030 con 0x7f40e80b5730 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e8150670 con 0x7f40e80b5730 2024-04-25T11:19:38.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e8050750 con 0x7f40e80b5730 2024-04-25T11:19:38.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40dffff640 1 -- 172.21.15.106:0/3837195056 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f40d801d070 con 0x7f40e80b5730 2024-04-25T11:19:38.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40dffff640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40c806f400 0x7f40c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40dffff640 1 -- 172.21.15.106:0/3837195056 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f40d808f340 con 0x7f40e80b5730 2024-04-25T11:19:38.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.953+0000 7f40e6ffd640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40c806f400 0x7f40c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.957+0000 7f40e6ffd640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40c806f400 0x7f40c8071910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f40d00af000 tx=0x7f40d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:38.961+0000 7f40dffff640 1 -- 172.21.15.106:0/3837195056 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f40d805bc70 con 0x7f40e80b5730 2024-04-25T11:19:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.105+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f40e8069390 con 0x7f40e80b5730 2024-04-25T11:19:39.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.105+0000 7f40dffff640 1 -- 172.21.15.106:0/3837195056 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f40d805ff70 con 0x7f40e80b5730 2024-04-25T11:19:39.105 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40c806f400 msgr2=0x7f40c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40c806f400 0x7f40c8071910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f40d00af000 tx=0x7f40d0008040 comp rx=0 tx=0).stop 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e80b5730 msgr2=0x7f40e81fd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e80b5730 0x7f40e81fd190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40d8007e60 tx=0x7f40d800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 shutdown_connections 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40e8160710 0x7f40e8157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e815fdc0 0x7f40e814d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f40c806f400 0x7f40c8071910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 --2- 172.21.15.106:0/3837195056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40e80b5730 0x7f40e81fd190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 >> 172.21.15.106:0/3837195056 conn(0x7f40e809ba30 msgr2=0x7f40e80b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 shutdown_connections 2024-04-25T11:19:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.109+0000 7f40eddb7640 1 -- 172.21.15.106:0/3837195056 wait complete. 2024-04-25T11:19:39.126 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.269+0000 7f3ff8ed4640 1 Processor -- start 2024-04-25T11:19:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.269+0000 7f3ff8ed4640 1 -- start start 2024-04-25T11:19:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff8ed4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ff4143fa0 0x7f3ff40cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff8ed4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff4073300 con 0x7f3ff40d1e50 2024-04-25T11:19:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff8ed4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff40735e0 con 0x7f3ff4072a20 2024-04-25T11:19:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff8ed4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40738c0 con 0x7f3ff4143fa0 2024-04-25T11:19:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3ff40d1e50 0x7f3ff40b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54282/0 (socket says 172.21.15.106:54282) 2024-04-25T11:19:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/3835223209 learned_addr learned my addr 172.21.15.106:0/3835223209 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff259c640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ff4143fa0 0x7f3ff40cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1858175414 0 0) 0x7f3ff4073300 con 0x7f3ff40d1e50 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fe0003530 con 0x7f3ff40d1e50 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 743505031 0 0) 0x7f3fe0003530 con 0x7f3ff40d1e50 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ff4143fa0 msgr2=0x7f3ff40cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ff4143fa0 0x7f3ff40cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 >> v1:172.21.15.134:6789/0 conn(0x7f3ff4072a20 legacy=0x7f3ff40d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40b5040 con 0x7f3ff40d1e50 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f3fec001a00 con 0x7f3ff40d1e50 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 >> v1:172.21.15.106:6789/0 conn(0x7f3ff40d1e50 legacy=0x7f3ff40b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe00039f0 0x7f3fe0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ff4143fa0 0x7f3fe0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe0004d50 0x7f3fe0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40738c0 con 0x7f3fe0004d50 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe000bb00 con 0x7f3fe00039f0 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe000bd00 con 0x7f3ff4143fa0 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff2d9d640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe0004d50 0x7f3fe0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff259c640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe00039f0 0x7f3fe0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3febfff640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ff4143fa0 0x7f3fe0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ff4143fa0 msgr2=0x7f3fe0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff2d9d640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ff4143fa0 0x7f3fe0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe00039f0 msgr2=0x7f3fe0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff2d9d640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe00039f0 0x7f3fe0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.273+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/3835223209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe0003530 con 0x7f3fe0004d50 2024-04-25T11:19:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff2d9d640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe0004d50 0x7f3fe0005160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00a830 tx=0x7f3fec01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b3aebf965ff0a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec00a480 con 0x7f3fe0004d50 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec0ad630 con 0x7f3fe0004d50 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff1d9b640 1 -- 172.21.15.106:0/3835223209 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0ada40 con 0x7f3fe0004d50 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/3835223209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe0004d50 msgr2=0x7f3fe0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff8ed4640 1 --2- 172.21.15.106:0/3835223209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe0004d50 0x7f3fe0005160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00a830 tx=0x7f3fec01d340 comp rx=0 tx=0).stop 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/3835223209 reap_dead start 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/3835223209 shutdown_connections 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/3835223209 >> 172.21.15.106:0/3835223209 conn(0x7f3ff406c9f0 msgr2=0x7f3ff406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/3835223209 shutdown_connections 2024-04-25T11:19:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/3835223209 wait complete. 2024-04-25T11:19:39.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff8ed4640 1 Processor -- start 2024-04-25T11:19:39.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.277+0000 7f3ff8ed4640 1 -- start start 2024-04-25T11:19:39.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fe0004d50 0x7f3ff41f9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff41609e0 0x7f3ff4135ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff40cc230 0x7f3ff40cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4072fb0 con 0x7f3ff40cc230 2024-04-25T11:19:39.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4072e30 con 0x7f3ff41609e0 2024-04-25T11:19:39.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40cd930 con 0x7f3fe0004d50 2024-04-25T11:19:39.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff40cc230 0x7f3ff40cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff40cc230 0x7f3ff40cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44650/0 (socket says 172.21.15.106:44650) 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/412820289 learned_addr learned my addr 172.21.15.106:0/412820289 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3febfff640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff41609e0 0x7f3ff4135ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff259c640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fe0004d50 0x7f3ff41f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/412820289 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fe0004d50 msgr2=0x7f3ff41f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fe0004d50 0x7f3ff41f9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/412820289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff41609e0 msgr2=0x7f3ff4135ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff41609e0 0x7f3ff4135ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 -- 172.21.15.106:0/412820289 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40ccb80 con 0x7f3ff40cc230 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff2d9d640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff40cc230 0x7f3ff40cc640 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3fec01d170 tx=0x7f3fe0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3feaffd640 1 -- 172.21.15.106:0/412820289 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0011d0 con 0x7f3ff40cc230 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3feaffd640 1 -- 172.21.15.106:0/412820289 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec0ad430 con 0x7f3ff40cc230 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff41e60c0 con 0x7f3ff40cc230 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3feaffd640 1 -- 172.21.15.106:0/412820289 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec001fe0 con 0x7f3ff40cc230 2024-04-25T11:19:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41fa2a0 con 0x7f3ff40cc230 2024-04-25T11:19:39.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff4050750 con 0x7f3ff40cc230 2024-04-25T11:19:39.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.281+0000 7f3feaffd640 1 -- 172.21.15.106:0/412820289 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3fec0be060 con 0x7f3ff40cc230 2024-04-25T11:19:39.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.285+0000 7f3feaffd640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3fd406f400 0x7f3fd4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.285+0000 7f3feaffd640 1 -- 172.21.15.106:0/412820289 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3fec1291a0 con 0x7f3ff40cc230 2024-04-25T11:19:39.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.285+0000 7f3ff259c640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3fd406f400 0x7f3fd4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.285+0000 7f3ff259c640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3fd406f400 0x7f3fd4071910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009970 tx=0x7f3fe4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.289+0000 7f3feaffd640 1 -- 172.21.15.106:0/412820289 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ff40ccb80 con 0x7f3ff40cc230 2024-04-25T11:19:39.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.433+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ff40b4e20 con 0x7f3ff40cc230 2024-04-25T11:19:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.433+0000 7f3feaffd640 1 -- 172.21.15.106:0/412820289 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3fec017050 con 0x7f3ff40cc230 2024-04-25T11:19:39.433 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:19:39.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3fd406f400 msgr2=0x7f3fd4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3fd406f400 0x7f3fd4071910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009970 tx=0x7f3fe4008040 comp rx=0 tx=0).stop 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff40cc230 msgr2=0x7f3ff40cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff40cc230 0x7f3ff40cc640 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3fec01d170 tx=0x7f3fe0005840 comp rx=0 tx=0).stop 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 shutdown_connections 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3fd406f400 0x7f3fd4071910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff40cc230 0x7f3ff40cc640 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff41609e0 0x7f3ff4135ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 --2- 172.21.15.106:0/412820289 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fe0004d50 0x7f3ff41f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 >> 172.21.15.106:0/412820289 conn(0x7f3ff406c9f0 msgr2=0x7f3ff4144da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 shutdown_connections 2024-04-25T11:19:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.437+0000 7f3ff8ed4640 1 -- 172.21.15.106:0/412820289 wait complete. 2024-04-25T11:19:39.453 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:39.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3c199640 1 Processor -- start 2024-04-25T11:19:39.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3c199640 1 -- start start 2024-04-25T11:19:39.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3c199640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1a340d1e50 0x7f1a3406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3c199640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1a3415d230 0x7f1a3415d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3c199640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a340b68e0 con 0x7f1a340d0cf0 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3c199640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1a34161210 con 0x7f1a3415d230 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3c199640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1a340b6290 con 0x7f1a340d1e50 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3a736640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1a3415d230 0x7f1a3415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3a736640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1a3415d230 0x7f1a3415d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44660/0 (socket says 172.21.15.106:44660) 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3a736640 1 -- 172.21.15.106:0/3298080471 learned_addr learned my addr 172.21.15.106:0/3298080471 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a39734640 1 --2- 172.21.15.106:0/3298080471 >> v2:172.21.15.134:3300/0 conn(0x7f1a340d1e50 0x7f1a3406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3a736640 1 -- 172.21.15.106:0/3298080471 >> v2:172.21.15.134:3300/0 conn(0x7f1a340d1e50 msgr2=0x7f1a3406ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3a736640 1 --2- 172.21.15.106:0/3298080471 >> v2:172.21.15.134:3300/0 conn(0x7f1a340d1e50 0x7f1a3406ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3a736640 1 -- 172.21.15.106:0/3298080471 >> v1:172.21.15.134:6789/0 conn(0x7f1a340d0cf0 legacy=0x7f1a340d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.593+0000 7f1a3a736640 1 -- 172.21.15.106:0/3298080471 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a340b6570 con 0x7f1a3415d230 2024-04-25T11:19:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3a736640 1 --2- 172.21.15.106:0/3298080471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a3415d230 0x7f1a3415d640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a340b1010 tx=0x7f1a2c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc48522daef8bdbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a38f33640 1 -- 172.21.15.106:0/3298080471 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0a5590 con 0x7f1a3415d230 2024-04-25T11:19:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a38f33640 1 -- 172.21.15.106:0/3298080471 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2c0a5730 con 0x7f1a3415d230 2024-04-25T11:19:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a38f33640 1 -- 172.21.15.106:0/3298080471 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0a5b00 con 0x7f1a3415d230 2024-04-25T11:19:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3c199640 1 -- 172.21.15.106:0/3298080471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a3415d230 msgr2=0x7f1a3415d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3c199640 1 --2- 172.21.15.106:0/3298080471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a3415d230 0x7f1a3415d640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a340b1010 tx=0x7f1a2c01c8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3c199640 1 -- 172.21.15.106:0/3298080471 shutdown_connections 2024-04-25T11:19:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3c199640 1 --2- 172.21.15.106:0/3298080471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a3415d230 0x7f1a3415d640 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3c199640 1 --2- 172.21.15.106:0/3298080471 >> v2:172.21.15.134:3300/0 conn(0x7f1a340d1e50 0x7f1a3406ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3c199640 1 -- 172.21.15.106:0/3298080471 >> 172.21.15.106:0/3298080471 conn(0x7f1a3406c9f0 msgr2=0x7f1a3406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3c199640 1 -- 172.21.15.106:0/3298080471 shutdown_connections 2024-04-25T11:19:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.597+0000 7f1a3c199640 1 -- 172.21.15.106:0/3298080471 wait complete. 2024-04-25T11:19:39.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 Processor -- start 2024-04-25T11:19:39.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 -- start start 2024-04-25T11:19:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a340d0cf0 0x7f1a3415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a3415d230 0x7f1a34161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a340b47a0 0x7f1a341555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3416a080 con 0x7f1a340b47a0 2024-04-25T11:19:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3415cdd0 con 0x7f1a3415d230 2024-04-25T11:19:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3416a200 con 0x7f1a340d0cf0 2024-04-25T11:19:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a340b47a0 0x7f1a341555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a340b47a0 0x7f1a341555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44666/0 (socket says 172.21.15.106:44666) 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 -- 172.21.15.106:0/1220469127 learned_addr learned my addr 172.21.15.106:0/1220469127 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a39734640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a3415d230 0x7f1a34161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a39f35640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a340d0cf0 0x7f1a3415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 -- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a340d0cf0 msgr2=0x7f1a3415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a340d0cf0 0x7f1a3415b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 -- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a3415d230 msgr2=0x7f1a34161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a3415d230 0x7f1a34161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 -- 172.21.15.106:0/1220469127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a340b4ef0 con 0x7f1a340b47a0 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a39734640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a3415d230 0x7f1a34161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a39f35640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a340d0cf0 0x7f1a3415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3a736640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a340b47a0 0x7f1a341555c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0194e0 tx=0x7f1a2c019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a2a7fc640 1 -- 172.21.15.106:0/1220469127 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0bc020 con 0x7f1a340b47a0 2024-04-25T11:19:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a2a7fc640 1 -- 172.21.15.106:0/1220469127 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2c0b8690 con 0x7f1a340b47a0 2024-04-25T11:19:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a340b51d0 con 0x7f1a340b47a0 2024-04-25T11:19:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a2a7fc640 1 -- 172.21.15.106:0/1220469127 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c009510 con 0x7f1a340b47a0 2024-04-25T11:19:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a34154dc0 con 0x7f1a340b47a0 2024-04-25T11:19:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.601+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34050750 con 0x7f1a340b47a0 2024-04-25T11:19:39.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.605+0000 7f1a2a7fc640 1 -- 172.21.15.106:0/1220469127 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1a2c0bf020 con 0x7f1a340b47a0 2024-04-25T11:19:39.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.605+0000 7f1a2a7fc640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1a1806f400 0x7f1a18071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.605+0000 7f1a2a7fc640 1 -- 172.21.15.106:0/1220469127 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a2c12a1d0 con 0x7f1a340b47a0 2024-04-25T11:19:39.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.605+0000 7f1a39f35640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1a1806f400 0x7f1a18071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.605+0000 7f1a39f35640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1a1806f400 0x7f1a18071910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1a340721f0 tx=0x7f1a30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.609+0000 7f1a2a7fc640 1 -- 172.21.15.106:0/1220469127 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a2c0c6030 con 0x7f1a340b47a0 2024-04-25T11:19:39.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1a3406d210 con 0x7f1a340b47a0 2024-04-25T11:19:39.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a2a7fc640 1 -- 172.21.15.106:0/1220469127 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1a2c0165e0 con 0x7f1a340b47a0 2024-04-25T11:19:39.755 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1a1806f400 msgr2=0x7f1a18071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1a1806f400 0x7f1a18071910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1a340721f0 tx=0x7f1a30009040 comp rx=0 tx=0).stop 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a340b47a0 msgr2=0x7f1a341555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a340b47a0 0x7f1a341555c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0194e0 tx=0x7f1a2c019670 comp rx=0 tx=0).stop 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 shutdown_connections 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1a1806f400 0x7f1a18071910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a340b47a0 0x7f1a341555c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a3415d230 0x7f1a34161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 --2- 172.21.15.106:0/1220469127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a340d0cf0 0x7f1a3415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 >> 172.21.15.106:0/1220469127 conn(0x7f1a3406c9f0 msgr2=0x7f1a340733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 shutdown_connections 2024-04-25T11:19:39.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.757+0000 7f1a3c199640 1 -- 172.21.15.106:0/1220469127 wait complete. 2024-04-25T11:19:39.776 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:39.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fab3b7640 1 Processor -- start 2024-04-25T11:19:39.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fab3b7640 1 -- start start 2024-04-25T11:19:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fab3b7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa40ce580 0x7f6fa4160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fab3b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6fa415dbc0 0x7f6fa4160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fab3b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa40b0be0 con 0x7f6fa4160bc0 2024-04-25T11:19:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fab3b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa4161510 con 0x7f6fa415dbc0 2024-04-25T11:19:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fab3b7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa40cecf0 con 0x7f6fa40ce580 2024-04-25T11:19:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fa9954640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6fa415dbc0 0x7f6fa4160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fa8952640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6fa4160bc0 0x7f6fa40b1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54298/0 (socket says 172.21.15.106:54298) 2024-04-25T11:19:39.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fa8952640 1 -- 172.21.15.106:0/2109335092 learned_addr learned my addr 172.21.15.106:0/2109335092 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:39.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fa9153640 1 --2- 172.21.15.106:0/2109335092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa40ce580 0x7f6fa4160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fa9954640 1 -- 172.21.15.106:0/2109335092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa40ce580 msgr2=0x7f6fa4160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:39.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fa9954640 1 --2- 172.21.15.106:0/2109335092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa40ce580 0x7f6fa4160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fa9954640 1 -- 172.21.15.106:0/2109335092 >> v1:172.21.15.106:6789/0 conn(0x7f6fa4160bc0 legacy=0x7f6fa40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6fa9954640 1 -- 172.21.15.106:0/2109335092 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa40b0ec0 con 0x7f6fa415dbc0 2024-04-25T11:19:39.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.917+0000 7f6f97fff640 1 -- 172.21.15.106:0/2109335092 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3507728857 0 0) 0x7f6fa40b0be0 con 0x7f6fa4160bc0 2024-04-25T11:19:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fa9954640 1 --2- 172.21.15.106:0/2109335092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa4160fd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40cee70 tx=0x7f6fa001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73aefa91faf8afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6f97fff640 1 -- 172.21.15.106:0/2109335092 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00ab620 con 0x7f6fa415dbc0 2024-04-25T11:19:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6f97fff640 1 -- 172.21.15.106:0/2109335092 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa00b0070 con 0x7f6fa415dbc0 2024-04-25T11:19:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6f97fff640 1 -- 172.21.15.106:0/2109335092 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00ab9e0 con 0x7f6fa415dbc0 2024-04-25T11:19:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fab3b7640 1 -- 172.21.15.106:0/2109335092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 msgr2=0x7f6fa4160fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/2109335092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa4160fd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40cee70 tx=0x7f6fa001c400 comp rx=0 tx=0).stop 2024-04-25T11:19:39.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fab3b7640 1 -- 172.21.15.106:0/2109335092 shutdown_connections 2024-04-25T11:19:39.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/2109335092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa4160fd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/2109335092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa40ce580 0x7f6fa4160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fab3b7640 1 -- 172.21.15.106:0/2109335092 >> 172.21.15.106:0/2109335092 conn(0x7f6fa40b78b0 msgr2=0x7f6fa40b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fab3b7640 1 -- 172.21.15.106:0/2109335092 shutdown_connections 2024-04-25T11:19:39.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.921+0000 7f6fab3b7640 1 -- 172.21.15.106:0/2109335092 wait complete. 2024-04-25T11:19:39.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 Processor -- start 2024-04-25T11:19:39.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 -- start start 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fa40ce580 0x7f6fa4143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa4160bc0 0x7f6fa415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4169a20 con 0x7f6fa415dbc0 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa40ce990 con 0x7f6fa40ce580 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa4169ba0 con 0x7f6fa4160bc0 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa415e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44682/0 (socket says 172.21.15.106:44682) 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 -- 172.21.15.106:0/3757828473 learned_addr learned my addr 172.21.15.106:0/3757828473 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 -- 172.21.15.106:0/3757828473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa4160bc0 msgr2=0x7f6fa415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa9954640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa4160bc0 0x7f6fa415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa9153640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fa40ce580 0x7f6fa4143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa4160bc0 0x7f6fa415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 -- 172.21.15.106:0/3757828473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fa40ce580 msgr2=0x7f6fa4143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fa40ce580 0x7f6fa4143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 -- 172.21.15.106:0/3757828473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa40d2a30 con 0x7f6fa415dbc0 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fa8952640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa415e210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c780 tx=0x7f6f9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6f95ffb640 1 -- 172.21.15.106:0/3757828473 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98018070 con 0x7f6fa415dbc0 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6f95ffb640 1 -- 172.21.15.106:0/3757828473 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9800a5e0 con 0x7f6fa415dbc0 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa415a190 con 0x7f6fa415dbc0 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6f95ffb640 1 -- 172.21.15.106:0/3757828473 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9800a9b0 con 0x7f6fa415dbc0 2024-04-25T11:19:39.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa415a750 con 0x7f6fa415dbc0 2024-04-25T11:19:39.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.925+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa40744a0 con 0x7f6fa415dbc0 2024-04-25T11:19:39.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.929+0000 7f6f95ffb640 1 -- 172.21.15.106:0/3757828473 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6f9801b070 con 0x7f6fa415dbc0 2024-04-25T11:19:39.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.929+0000 7f6f95ffb640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f8806f400 0x7f6f88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.929+0000 7f6fa9153640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f8806f400 0x7f6f88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.929+0000 7f6f95ffb640 1 -- 172.21.15.106:0/3757828473 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6f9808e870 con 0x7f6fa415dbc0 2024-04-25T11:19:39.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.929+0000 7f6fa9153640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f8806f400 0x7f6f88071910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0096f0 tx=0x7f6f9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:39.933+0000 7f6f95ffb640 1 -- 172.21.15.106:0/3757828473 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f9805b7f0 con 0x7f6fa415dbc0 2024-04-25T11:19:40.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.113+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fa4159bb0 con 0x7f6fa415dbc0 2024-04-25T11:19:40.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6f95ffb640 1 -- 172.21.15.106:0/3757828473 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f6f9805b0f0 con 0x7f6fa415dbc0 2024-04-25T11:19:40.114 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:40.114 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:26.806158+0000","modified":"2024-04-25T11:19:29.823480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:19:40.114 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T11:19:40.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f8806f400 msgr2=0x7f6f88071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f8806f400 0x7f6f88071910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0096f0 tx=0x7f6f9c008040 comp rx=0 tx=0).stop 2024-04-25T11:19:40.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 msgr2=0x7f6fa415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa415e210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c780 tx=0x7f6f9800cc40 comp rx=0 tx=0).stop 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 shutdown_connections 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fa4160bc0 0x7f6fa415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f8806f400 0x7f6f88071910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa415dbc0 0x7f6fa415e210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 --2- 172.21.15.106:0/3757828473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fa40ce580 0x7f6fa4143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 >> 172.21.15.106:0/3757828473 conn(0x7f6fa40b78b0 msgr2=0x7f6fa4158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 shutdown_connections 2024-04-25T11:19:40.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.117+0000 7f6fab3b7640 1 -- 172.21.15.106:0/3757828473 wait complete. 2024-04-25T11:19:40.134 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:40.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2cc1cb640 1 Processor -- start 2024-04-25T11:19:40.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2cc1cb640 1 -- start start 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2cc1cb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa2c40b32d0 0x7fa2c40b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2cc1cb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa2c40ce120 0x7fa2c40ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2cc1cb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c40afe20 con 0x7fa2c40af300 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2cc1cb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c40b1e60 con 0x7fa2c40b32d0 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2cc1cb640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c40b3ae0 con 0x7fa2c40ce120 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2c9f67640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa2c40b32d0 0x7fa2c40b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2ca768640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa2c40af300 0x7fa2c40af710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54300/0 (socket says 172.21.15.106:54300) 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2ca768640 1 -- 172.21.15.106:0/2762163817 learned_addr learned my addr 172.21.15.106:0/2762163817 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2c9766640 1 --2- 172.21.15.106:0/2762163817 >> v2:172.21.15.134:3300/0 conn(0x7fa2c40ce120 0x7fa2c40ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2c9f67640 1 -- 172.21.15.106:0/2762163817 >> v2:172.21.15.134:3300/0 conn(0x7fa2c40ce120 msgr2=0x7fa2c40ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:40.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2c9f67640 1 --2- 172.21.15.106:0/2762163817 >> v2:172.21.15.134:3300/0 conn(0x7fa2c40ce120 0x7fa2c40ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2c9f67640 1 -- 172.21.15.106:0/2762163817 >> v1:172.21.15.106:6789/0 conn(0x7fa2c40af300 legacy=0x7fa2c40af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.285+0000 7fa2c9f67640 1 -- 172.21.15.106:0/2762163817 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c40b2100 con 0x7fa2c40b32d0 2024-04-25T11:19:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2c9f67640 1 --2- 172.21.15.106:0/2762163817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40b32d0 0x7fa2c40b2900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4073d80 tx=0x7fa2c001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3622e96f83b716e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2c8f65640 1 -- 172.21.15.106:0/2762163817 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00a7070 con 0x7fa2c40b32d0 2024-04-25T11:19:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2c8f65640 1 -- 172.21.15.106:0/2762163817 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c00b0070 con 0x7fa2c40b32d0 2024-04-25T11:19:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2c8f65640 1 -- 172.21.15.106:0/2762163817 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00ab890 con 0x7fa2c40b32d0 2024-04-25T11:19:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/2762163817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40b32d0 msgr2=0x7fa2c40b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/2762163817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40b32d0 0x7fa2c40b2900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4073d80 tx=0x7fa2c001c450 comp rx=0 tx=0).stop 2024-04-25T11:19:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/2762163817 shutdown_connections 2024-04-25T11:19:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/2762163817 >> v2:172.21.15.134:3300/0 conn(0x7fa2c40ce120 0x7fa2c40ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/2762163817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40b32d0 0x7fa2c40b2900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/2762163817 >> 172.21.15.106:0/2762163817 conn(0x7fa2c40b4020 msgr2=0x7fa2c40b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/2762163817 shutdown_connections 2024-04-25T11:19:40.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.289+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/2762163817 wait complete. 2024-04-25T11:19:40.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 Processor -- start 2024-04-25T11:19:40.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 -- start start 2024-04-25T11:19:40.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40af300 0x7fa2c415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40b32d0 0x7fa2c4166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40ce120 0x7fa2c4167230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c40b36e0 con 0x7fa2c40ce120 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c40cea70 con 0x7fa2c40af300 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c40b3860 con 0x7fa2c40b32d0 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40ce120 0x7fa2c4167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40ce120 0x7fa2c4167230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44704/0 (socket says 172.21.15.106:44704) 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 -- 172.21.15.106:0/689929138 learned_addr learned my addr 172.21.15.106:0/689929138 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2c9766640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40b32d0 0x7fa2c4166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2c9f67640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40af300 0x7fa2c415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 -- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40b32d0 msgr2=0x7fa2c4166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40b32d0 0x7fa2c4166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 -- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40af300 msgr2=0x7fa2c415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40af300 0x7fa2c415cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 -- 172.21.15.106:0/689929138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c41659c0 con 0x7fa2c40ce120 2024-04-25T11:19:40.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2c9f67640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40af300 0x7fa2c415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:40.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ca768640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40ce120 0x7fa2c4167230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00cad0 tx=0x7fa2bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ba7fc640 1 -- 172.21.15.106:0/689929138 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc017070 con 0x7fa2c40ce120 2024-04-25T11:19:40.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ba7fc640 1 -- 172.21.15.106:0/689929138 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00ac30 con 0x7fa2c40ce120 2024-04-25T11:19:40.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2ba7fc640 1 -- 172.21.15.106:0/689929138 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc012400 con 0x7fa2c40ce120 2024-04-25T11:19:40.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c40b0750 con 0x7fa2c40ce120 2024-04-25T11:19:40.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c409e9c0 con 0x7fa2c40ce120 2024-04-25T11:19:40.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.293+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c406eb80 con 0x7fa2c40ce120 2024-04-25T11:19:40.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.297+0000 7fa2ba7fc640 1 -- 172.21.15.106:0/689929138 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa2bc01a070 con 0x7fa2c40ce120 2024-04-25T11:19:40.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.297+0000 7fa2ba7fc640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa2a806f400 0x7fa2a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.297+0000 7fa2c9f67640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa2a806f400 0x7fa2a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.297+0000 7fa2ba7fc640 1 -- 172.21.15.106:0/689929138 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa2bc08f6a0 con 0x7fa2c40ce120 2024-04-25T11:19:40.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.297+0000 7fa2c9f67640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa2a806f400 0x7fa2a8071910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00af000 tx=0x7fa2c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.301+0000 7fa2ba7fc640 1 -- 172.21.15.106:0/689929138 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa2c41659c0 con 0x7fa2c40ce120 2024-04-25T11:19:40.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.461+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2c415d160 con 0x7fa2c40ce120 2024-04-25T11:19:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2ba7fc640 1 -- 172.21.15.106:0/689929138 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7fa2bc05c020 con 0x7fa2c40ce120 2024-04-25T11:19:40.463 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:40.463 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa2a806f400 msgr2=0x7fa2a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa2a806f400 0x7fa2a8071910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00af000 tx=0x7fa2c0008040 comp rx=0 tx=0).stop 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40ce120 msgr2=0x7fa2c4167230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40ce120 0x7fa2c4167230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00cad0 tx=0x7fa2bc00cf90 comp rx=0 tx=0).stop 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 shutdown_connections 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa2a806f400 0x7fa2a8071910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2c40ce120 0x7fa2c4167230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40b32d0 0x7fa2c4166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 --2- 172.21.15.106:0/689929138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40af300 0x7fa2c415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 >> 172.21.15.106:0/689929138 conn(0x7fa2c40b4020 msgr2=0x7fa2c40b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 shutdown_connections 2024-04-25T11:19:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.465+0000 7fa2cc1cb640 1 -- 172.21.15.106:0/689929138 wait complete. 2024-04-25T11:19:40.485 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff245de4640 1 Processor -- start 2024-04-25T11:19:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff245de4640 1 -- start start 2024-04-25T11:19:40.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff245de4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 0x7ff2400d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff245de4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff240072a50 0x7ff240072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff245de4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2400733a0 con 0x7ff2400b43f0 2024-04-25T11:19:40.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff245de4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff24015fdb0 con 0x7ff240072a50 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff245de4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff2400b49b0 con 0x7ff24015c270 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff23ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff240072a50 0x7ff240072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff23ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff240072a50 0x7ff240072e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44720/0 (socket says 172.21.15.106:44720) 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff23ffff640 1 -- 172.21.15.106:0/1470466108 learned_addr learned my addr 172.21.15.106:0/1470466108 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff23f7fe640 1 --2- 172.21.15.106:0/1470466108 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 0x7ff2400d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff23ffff640 1 -- 172.21.15.106:0/1470466108 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 msgr2=0x7ff2400d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff23ffff640 1 --2- 172.21.15.106:0/1470466108 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 0x7ff2400d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff23ffff640 1 -- 172.21.15.106:0/1470466108 >> v1:172.21.15.134:6789/0 conn(0x7ff2400b43f0 legacy=0x7ff2400cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.633+0000 7ff23ffff640 1 -- 172.21.15.106:0/1470466108 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2400b4c90 con 0x7ff240072a50 2024-04-25T11:19:40.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff23ffff640 1 --2- 172.21.15.106:0/1470466108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff240072a50 0x7ff240072e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff228009870 tx=0x7ff22801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276227709c8ea3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff23e7fc640 1 -- 172.21.15.106:0/1470466108 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2280b0070 con 0x7ff240072a50 2024-04-25T11:19:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff23e7fc640 1 -- 172.21.15.106:0/1470466108 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2280ab600 con 0x7ff240072a50 2024-04-25T11:19:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff23e7fc640 1 -- 172.21.15.106:0/1470466108 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2280ab9d0 con 0x7ff240072a50 2024-04-25T11:19:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 -- 172.21.15.106:0/1470466108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff240072a50 msgr2=0x7ff240072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 --2- 172.21.15.106:0/1470466108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff240072a50 0x7ff240072e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff228009870 tx=0x7ff22801c560 comp rx=0 tx=0).stop 2024-04-25T11:19:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 -- 172.21.15.106:0/1470466108 shutdown_connections 2024-04-25T11:19:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 --2- 172.21.15.106:0/1470466108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff240072a50 0x7ff240072e60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 --2- 172.21.15.106:0/1470466108 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 0x7ff2400d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 -- 172.21.15.106:0/1470466108 >> 172.21.15.106:0/1470466108 conn(0x7ff24006c7c0 msgr2=0x7ff24006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 -- 172.21.15.106:0/1470466108 shutdown_connections 2024-04-25T11:19:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 -- 172.21.15.106:0/1470466108 wait complete. 2024-04-25T11:19:40.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 Processor -- start 2024-04-25T11:19:40.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.637+0000 7ff245de4640 1 -- start start 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff240072a50 0x7ff240165c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2400b43f0 0x7ff240149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 0x7ff2400cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff2400d1880 con 0x7ff2400b43f0 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2400cf3d0 con 0x7ff240072a50 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff2400d1a00 con 0x7ff24015c270 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2400b43f0 0x7ff240149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2400b43f0 0x7ff240149e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44732/0 (socket says 172.21.15.106:44732) 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 -- 172.21.15.106:0/2148681912 learned_addr learned my addr 172.21.15.106:0/2148681912 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23f7fe640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff240072a50 0x7ff240165c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23ffff640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 0x7ff2400cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 -- 172.21.15.106:0/2148681912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 msgr2=0x7ff2400cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 0x7ff2400cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 -- 172.21.15.106:0/2148681912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff240072a50 msgr2=0x7ff240165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff240072a50 0x7ff240165c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 -- 172.21.15.106:0/2148681912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2400aa490 con 0x7ff2400b43f0 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff23effd640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2400b43f0 0x7ff240149e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c760 tx=0x7ff23000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff237fff640 1 -- 172.21.15.106:0/2148681912 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230003ba0 con 0x7ff2400b43f0 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff237fff640 1 -- 172.21.15.106:0/2148681912 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff230003d60 con 0x7ff2400b43f0 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2400aa990 con 0x7ff2400b43f0 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff237fff640 1 -- 172.21.15.106:0/2148681912 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230005310 con 0x7ff2400b43f0 2024-04-25T11:19:40.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24015cb20 con 0x7ff2400b43f0 2024-04-25T11:19:40.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24006ccd0 con 0x7ff2400b43f0 2024-04-25T11:19:40.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.641+0000 7ff237fff640 1 -- 172.21.15.106:0/2148681912 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2300054b0 con 0x7ff2400b43f0 2024-04-25T11:19:40.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.645+0000 7ff237fff640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff22006f400 0x7ff220071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.645+0000 7ff23f7fe640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff22006f400 0x7ff220071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.645+0000 7ff237fff640 1 -- 172.21.15.106:0/2148681912 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff23008f0d0 con 0x7ff2400b43f0 2024-04-25T11:19:40.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.645+0000 7ff23f7fe640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff22006f400 0x7ff220071910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff23800b930 tx=0x7ff238008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.649+0000 7ff237fff640 1 -- 172.21.15.106:0/2148681912 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff2400733a0 con 0x7ff2400b43f0 2024-04-25T11:19:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.825+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2400aa720 con 0x7ff2400b43f0 2024-04-25T11:19:40.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff237fff640 1 -- 172.21.15.106:0/2148681912 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7ff23005c050 con 0x7ff2400b43f0 2024-04-25T11:19:40.827 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:40.827 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:26.806158+0000","modified":"2024-04-25T11:19:29.823480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:19:40.827 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff22006f400 msgr2=0x7ff220071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff22006f400 0x7ff220071910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff23800b930 tx=0x7ff238008040 comp rx=0 tx=0).stop 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2400b43f0 msgr2=0x7ff240149e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2400b43f0 0x7ff240149e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c760 tx=0x7ff23000cb60 comp rx=0 tx=0).stop 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 shutdown_connections 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff24015c270 0x7ff2400cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff22006f400 0x7ff220071910 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2400b43f0 0x7ff240149e30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 --2- 172.21.15.106:0/2148681912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff240072a50 0x7ff240165c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 >> 172.21.15.106:0/2148681912 conn(0x7ff24006c7c0 msgr2=0x7ff240167830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 shutdown_connections 2024-04-25T11:19:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:40.829+0000 7ff245de4640 1 -- 172.21.15.106:0/2148681912 wait complete. 2024-04-25T11:19:41.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:19:26.806158+0000', 'modified': '2024-04-25T11:19:29.823480+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6833/3507437799', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 3507437799}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 3507437799}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-25T11:19:41.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:19:41.250 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T11:19:41.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb7897d8640 1 Processor -- start 2024-04-25T11:19:41.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb7897d8640 1 -- start start 2024-04-25T11:19:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb7897d8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb78415fd10 0x7fb7840b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb7897d8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840d1350 con 0x7fb7840d2150 2024-04-25T11:19:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb7897d8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840b9270 con 0x7fb7840d44a0 2024-04-25T11:19:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb7897d8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb784163cc0 con 0x7fb78415fd10 2024-04-25T11:19:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb7827fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb7840d2150 0x7fb7840d2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54310/0 (socket says 172.21.15.106:54310) 2024-04-25T11:19:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb7827fc640 1 -- 172.21.15.106:0/2623055534 learned_addr learned my addr 172.21.15.106:0/2623055534 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb782ffd640 1 --2- 172.21.15.106:0/2623055534 >> v2:172.21.15.134:3300/0 conn(0x7fb78415fd10 0x7fb7840b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771595757 0 0) 0x7fb7840d1350 con 0x7fb7840d2150 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.413+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb770003530 con 0x7fb7840d2150 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1676659143 0 0) 0x7fb770003530 con 0x7fb7840d2150 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 >> v2:172.21.15.134:3300/0 conn(0x7fb78415fd10 msgr2=0x7fb7840b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 --2- 172.21.15.106:0/2623055534 >> v2:172.21.15.134:3300/0 conn(0x7fb78415fd10 0x7fb7840b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 >> v1:172.21.15.134:6789/0 conn(0x7fb7840d44a0 legacy=0x7fb7840d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb784163fa0 con 0x7fb7840d2150 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fb76c001a00 con 0x7fb7840d2150 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 >> v1:172.21.15.106:6789/0 conn(0x7fb7840d2150 legacy=0x7fb7840d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700039c0 0x7fb770003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770004810 0x7fb770004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7700051d0 0x7fb77000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb784163cc0 con 0x7fb7700051d0 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb77000c170 con 0x7fb770004810 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb77000c370 con 0x7fb7700039c0 2024-04-25T11:19:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7837fe640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7700051d0 0x7fb77000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb782ffd640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700039c0 0x7fb770003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7827fc640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770004810 0x7fb770004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7837fe640 1 -- 172.21.15.106:0/2623055534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700039c0 msgr2=0x7fb770003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7837fe640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700039c0 0x7fb770003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7837fe640 1 -- 172.21.15.106:0/2623055534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770004810 msgr2=0x7fb770004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7837fe640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770004810 0x7fb770004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7837fe640 1 -- 172.21.15.106:0/2623055534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb770003530 con 0x7fb7700051d0 2024-04-25T11:19:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7837fe640 1 -- 172.21.15.106:0/2623055534 reap_dead start 2024-04-25T11:19:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7837fe640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7700051d0 0x7fb77000b9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001780 tx=0x7fb77c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c270a3abdc04e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0013a0 con 0x7fb7700051d0 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c005310 con 0x7fb7700051d0 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb781ffb640 1 -- 172.21.15.106:0/2623055534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0054c0 con 0x7fb7700051d0 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7897d8640 1 -- 172.21.15.106:0/2623055534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7700051d0 msgr2=0x7fb77000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7897d8640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7700051d0 0x7fb77000b9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001780 tx=0x7fb77c01c360 comp rx=0 tx=0).stop 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7897d8640 1 -- 172.21.15.106:0/2623055534 shutdown_connections 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7897d8640 1 --2- 172.21.15.106:0/2623055534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7700051d0 0x7fb77000b9d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7897d8640 1 -- 172.21.15.106:0/2623055534 >> 172.21.15.106:0/2623055534 conn(0x7fb78406d7f0 msgr2=0x7fb78406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7897d8640 1 -- 172.21.15.106:0/2623055534 shutdown_connections 2024-04-25T11:19:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.417+0000 7fb7897d8640 1 -- 172.21.15.106:0/2623055534 wait complete. 2024-04-25T11:19:41.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7897d8640 1 Processor -- start 2024-04-25T11:19:41.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7897d8640 1 -- start start 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7897d8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700051d0 0x7fb7841fa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7897d8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784156050 0x7fb78415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7897d8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78415cb50 0x7fb784156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7897d8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb784160120 con 0x7fb78415cb50 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7897d8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb784164560 con 0x7fb784156050 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7897d8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb7841638e0 con 0x7fb7700051d0 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78415cb50 0x7fb784156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78415cb50 0x7fb784156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44754/0 (socket says 172.21.15.106:44754) 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 -- 172.21.15.106:0/842510010 learned_addr learned my addr 172.21.15.106:0/842510010 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7827fc640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784156050 0x7fb78415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb782ffd640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700051d0 0x7fb7841fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 -- 172.21.15.106:0/842510010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700051d0 msgr2=0x7fb7841fa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700051d0 0x7fb7841fa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 -- 172.21.15.106:0/842510010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784156050 msgr2=0x7fb78415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784156050 0x7fb78415c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 -- 172.21.15.106:0/842510010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78415cf60 con 0x7fb78415cb50 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb7837fe640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78415cb50 0x7fb784156810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb77c01c190 tx=0x7fb770005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.421+0000 7fb77b7fe640 1 -- 172.21.15.106:0/842510010 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0013a0 con 0x7fb78415cb50 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb77b7fe640 1 -- 172.21.15.106:0/842510010 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c0050d0 con 0x7fb78415cb50 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb77b7fe640 1 -- 172.21.15.106:0/842510010 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c004ed0 con 0x7fb78415cb50 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb7897d8640 1 -- 172.21.15.106:0/842510010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7841fa680 con 0x7fb78415cb50 2024-04-25T11:19:41.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb7897d8640 1 -- 172.21.15.106:0/842510010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb784157f30 con 0x7fb78415cb50 2024-04-25T11:19:41.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb77b7fe640 1 -- 172.21.15.106:0/842510010 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb77c007440 con 0x7fb78415cb50 2024-04-25T11:19:41.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb77b7fe640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb76406f400 0x7fb764071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb77b7fe640 1 -- 172.21.15.106:0/842510010 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb77c1282f0 con 0x7fb78415cb50 2024-04-25T11:19:41.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb782ffd640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb76406f400 0x7fb764071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb782ffd640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb76406f400 0x7fb764071910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7740099b0 tx=0x7fb774008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb790b35000 1 Processor -- start 2024-04-25T11:19:41.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.425+0000 7fb790b35000 1 -- start start 2024-04-25T11:19:41.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 0x55fc76ca6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b34ea0 0x55fc76ca7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76ca7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fc76b16160 con 0x55fc7672d280 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fc76aabe20 con 0x55fc76b34ea0 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fc76a64a20 con 0x55fc76b48c20 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76ca7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76ca7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44766/0 (socket says 172.21.15.106:44766) 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 -- 172.21.15.106:0/1709401841 learned_addr learned my addr 172.21.15.106:0/1709401841 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7827fc640 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b34ea0 0x55fc76ca7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb782ffd640 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 0x55fc76ca6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 -- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 msgr2=0x55fc76ca6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 0x55fc76ca6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 -- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b34ea0 msgr2=0x55fc76ca7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b34ea0 0x55fc76ca7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 -- 172.21.15.106:0/1709401841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc76a69100 con 0x55fc7672d280 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7827fc640 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b34ea0 0x55fc76ca7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb7837fe640 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76ca7960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb77c126840 tx=0x7fb77c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf6864431fbb0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb778ff9640 1 -- 172.21.15.106:0/1709401841 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c10e0d0 con 0x55fc7672d280 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb778ff9640 1 -- 172.21.15.106:0/1709401841 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c0fd290 con 0x55fc7672d280 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb778ff9640 1 -- 172.21.15.106:0/1709401841 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0f4ae0 con 0x55fc7672d280 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- 172.21.15.106:0/1709401841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 msgr2=0x55fc76ca7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76ca7960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb77c126840 tx=0x7fb77c007a40 comp rx=0 tx=0).stop 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- 172.21.15.106:0/1709401841 shutdown_connections 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76ca7960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b34ea0 0x55fc76ca7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 --2- 172.21.15.106:0/1709401841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 0x55fc76ca6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- 172.21.15.106:0/1709401841 >> 172.21.15.106:0/1709401841 conn(0x55fc7676cff0 msgr2=0x55fc76c5c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- 172.21.15.106:0/1709401841 shutdown_connections 2024-04-25T11:19:41.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- 172.21.15.106:0/1709401841 wait complete. 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.429+0000 7fb790b35000 1 -- start start 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 Processor -- start 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc7672d280 0x55fc76cadc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 0x55fc76c690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fc76ab9220 con 0x55fc76b34ea0 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fc76a4fea0 con 0x55fc7672d280 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fc76aa6b00 con 0x55fc76b48c20 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 10 client.0 _ll_register_callbacks cb 0x55fc76ab8dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:19:41.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44778/0 (socket says 172.21.15.106:44778) 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 -- 172.21.15.106:0/539262040 learned_addr learned my addr 172.21.15.106:0/539262040 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7837fe640 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 0x55fc76c690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb782ffd640 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc7672d280 0x55fc76cadc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 -- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 msgr2=0x55fc76c690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 0x55fc76c690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 -- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc7672d280 msgr2=0x55fc76cadc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc7672d280 0x55fc76cadc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 -- 172.21.15.106:0/539262040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc76a4b500 con 0x55fc76b34ea0 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb782ffd640 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc7672d280 0x55fc76cadc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb7827fc640 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cae190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb76c002240 tx=0x7fb76c0078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30a8740a444e594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb761ffb640 1 -- 172.21.15.106:0/539262040 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c019020 con 0x55fc76b34ea0 2024-04-25T11:19:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb761ffb640 1 -- 172.21.15.106:0/539262040 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb76c011bb0 con 0x55fc76b34ea0 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 -- 172.21.15.106:0/539262040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77c0fd290 con 0x55fc76b34ea0 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb761ffb640 1 -- 172.21.15.106:0/539262040 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb76c023050 con 0x55fc76b34ea0 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 10 client.4611 fetch_fsmap learned FSMap version 8 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 -- 172.21.15.106:0/539262040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55fc76a31ea0 con 0x55fc76b34ea0 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb761ffb640 1 -- 172.21.15.106:0/539262040 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c0fd290 con 0x55fc76b34ea0 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb761ffb640 1 -- 172.21.15.106:0/539262040 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fb76c026020 con 0x55fc76b34ea0 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 10 client.4611 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 10 client.4611 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 20 client.4611 populate_metadata read hostname 'smithi106' 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] conn(0x55fc767517d0 0x55fc76c69e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 4 client.4611 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb790b35000 1 -- 172.21.15.106:0/539262040 --> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc76a44be0 con 0x55fc767517d0 2024-04-25T11:19:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb782ffd640 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] conn(0x55fc767517d0 0x55fc76c69e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb782ffd640 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] conn(0x55fc767517d0 0x55fc76c69e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5536b12f1e7ba297 server_cookie=7aa68455a9dc08ea in_seq=0 out_seq=0 2024-04-25T11:19:41.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.433+0000 7fb761ffb640 10 client.4611 ms_handle_connect on v2:172.21.15.134:6832/3507437799 2024-04-25T11:19:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb761ffb640 1 -- 172.21.15.106:0/539262040 <== mds.0 v2:172.21.15.134:6832/3507437799 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc76a44be0 con 0x55fc767517d0 2024-04-25T11:19:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb761ffb640 10 client.4611 handle_command_reply: tid=0 2024-04-25T11:19:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 client.4611 shutdown 2024-04-25T11:19:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 -- 172.21.15.106:0/539262040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 msgr2=0x55fc76cae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cae190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb76c002240 tx=0x7fb76c0078d0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 -- 172.21.15.106:0/539262040 shutdown_connections 2024-04-25T11:19:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] conn(0x55fc767517d0 0x55fc76c69e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b48c20 0x55fc76c690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cae190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 --2- 172.21.15.106:0/539262040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc7672d280 0x55fc76cadc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb761ffb640 0 client.4611 ms_handle_reset on v2:172.21.15.134:6832/3507437799 2024-04-25T11:19:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 -- 172.21.15.106:0/539262040 >> 172.21.15.106:0/539262040 conn(0x55fc7676cff0 msgr2=0x55fc76c5cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 -- 172.21.15.106:0/539262040 shutdown_connections 2024-04-25T11:19:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 1 -- 172.21.15.106:0/539262040 wait complete. 2024-04-25T11:19:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.437+0000 7fb790b35000 20 client.4611 trim_cache size 0 max 16384 2024-04-25T11:19:41.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.453+0000 7fb790b35000 1 Processor -- start 2024-04-25T11:19:41.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.453+0000 7fb790b35000 1 -- start start 2024-04-25T11:19:41.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc7672d280 0x55fc76cb32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cbcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 0x55fc76cbd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb784201ee0 con 0x55fc76b34ea0 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840ba4b0 con 0x55fc76b48c20 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb7840bab00 con 0x55fc7672d280 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cbcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cbcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44782/0 (socket says 172.21.15.106:44782) 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 -- 172.21.15.106:0/317121207 learned_addr learned my addr 172.21.15.106:0/317121207 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 -- 172.21.15.106:0/317121207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc7672d280 msgr2=0x55fc76cb32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 --2- 172.21.15.106:0/317121207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc7672d280 0x55fc76cb32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7837fe640 1 --2- 172.21.15.106:0/317121207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 0x55fc76cbd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 -- 172.21.15.106:0/317121207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 msgr2=0x55fc76cbd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 --2- 172.21.15.106:0/317121207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 0x55fc76cbd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 -- 172.21.15.106:0/317121207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc76c60840 con 0x55fc76b34ea0 2024-04-25T11:19:41.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb7827fc640 1 --2- 172.21.15.106:0/317121207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cbcde0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009010 tx=0x7fb76c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0469c28f6559564 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb763fff640 1 -- 172.21.15.106:0/317121207 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c019030 con 0x55fc76b34ea0 2024-04-25T11:19:41.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb763fff640 1 -- 172.21.15.106:0/317121207 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c011bd0 con 0x55fc76b34ea0 2024-04-25T11:19:41.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb763fff640 1 -- 172.21.15.106:0/317121207 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c010040 con 0x55fc76b34ea0 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- 172.21.15.106:0/317121207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 msgr2=0x55fc76cbcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 --2- 172.21.15.106:0/317121207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cbcde0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009010 tx=0x7fb76c015000 comp rx=0 tx=0).stop 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- 172.21.15.106:0/317121207 shutdown_connections 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 --2- 172.21.15.106:0/317121207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 0x55fc76cbd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 --2- 172.21.15.106:0/317121207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc76b34ea0 0x55fc76cbcde0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 --2- 172.21.15.106:0/317121207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc7672d280 0x55fc76cb32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- 172.21.15.106:0/317121207 >> 172.21.15.106:0/317121207 conn(0x55fc7676cff0 msgr2=0x55fc76caca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- 172.21.15.106:0/317121207 shutdown_connections 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- 172.21.15.106:0/317121207 wait complete. 2024-04-25T11:19:41.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 -- start start 2024-04-25T11:19:41.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.457+0000 7fb790b35000 1 Processor -- start 2024-04-25T11:19:41.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76d19380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b34ea0 0x55fc76d198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 0x55fc76cb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fc76aab130 con 0x55fc7672d280 2024-04-25T11:19:41.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fc76a417d0 con 0x55fc76b48c20 2024-04-25T11:19:41.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fc76ab0450 con 0x55fc76b34ea0 2024-04-25T11:19:41.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 10 client.0 _ll_register_callbacks cb 0x7fb76c001fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76d19380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76d19380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44786/0 (socket says 172.21.15.106:44786) 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 -- 172.21.15.106:0/454489829 learned_addr learned my addr 172.21.15.106:0/454489829 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 -- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b34ea0 msgr2=0x55fc76d198c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b34ea0 0x55fc76d198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb7837fe640 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 0x55fc76cb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 -- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 msgr2=0x55fc76cb1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 0x55fc76cb1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 -- 172.21.15.106:0/454489829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc76c69430 con 0x55fc7672d280 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb782ffd640 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76d19380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7740136b0 tx=0x7fb774009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92cc1264dc901b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 1 -- 172.21.15.106:0/454489829 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774004020 con 0x55fc7672d280 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 1 -- 172.21.15.106:0/454489829 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7740041c0 con 0x55fc7672d280 2024-04-25T11:19:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 -- 172.21.15.106:0/454489829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76c011bd0 con 0x55fc7672d280 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 1 -- 172.21.15.106:0/454489829 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb774039050 con 0x55fc7672d280 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 10 client.4617 fetch_fsmap learned FSMap version 8 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 -- 172.21.15.106:0/454489829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55fc76a64c70 con 0x55fc7672d280 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 1 -- 172.21.15.106:0/454489829 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c011bd0 con 0x55fc7672d280 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 1 -- 172.21.15.106:0/454489829 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fb77403c020 con 0x55fc7672d280 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 10 client.4617 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 10 client.4617 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 20 client.4617 populate_metadata read hostname 'smithi106' 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] conn(0x55fc7673f830 0x55fc76d221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 4 client.4617 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 -- 172.21.15.106:0/454489829 --> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] -- command(tid 0: {"prefix": "status"}) -- 0x55fc76ab9a30 con 0x55fc7673f830 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb7827fc640 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] conn(0x55fc7673f830 0x55fc76d221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 10 client.4617 ms_handle_connect on v2:172.21.15.134:6832/3507437799 2024-04-25T11:19:41.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb7827fc640 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] conn(0x55fc7673f830 0x55fc76d221f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db2213e9480daf43 server_cookie=c00207f20aaa7765 in_seq=0 out_seq=0 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 1 -- 172.21.15.106:0/454489829 <== mds.0 v2:172.21.15.134:6832/3507437799 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55fc76ab9a30 con 0x55fc7673f830 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb761ffb640 10 client.4617 handle_command_reply: tid=0 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.461+0000 7fb790b35000 1 client.4617 shutdown 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 -- 172.21.15.106:0/454489829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 msgr2=0x55fc76d19380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76d19380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7740136b0 tx=0x7fb774009190 comp rx=0 tx=0).stop 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 -- 172.21.15.106:0/454489829 shutdown_connections 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:6832/3507437799,v1:172.21.15.134:6833/3507437799] conn(0x55fc7673f830 0x55fc76d221f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc76b48c20 0x55fc76cb1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb761ffb640 0 client.4617 ms_handle_reset on v2:172.21.15.134:6832/3507437799 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fc76b34ea0 0x55fc76d198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 --2- 172.21.15.106:0/454489829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc7672d280 0x55fc76d19380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 -- 172.21.15.106:0/454489829 >> 172.21.15.106:0/454489829 conn(0x55fc7676cff0 msgr2=0x55fc76c635d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 -- 172.21.15.106:0/454489829 shutdown_connections 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 1 -- 172.21.15.106:0/454489829 wait complete. 2024-04-25T11:19:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb790b35000 20 client.4617 trim_cache size 0 max 16384 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "8efdb3a5-72cb-436f-9be5-cf4e40adf704", 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4285, 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 14.044986075000001, 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 8, 2024-04-25T11:19:41.466 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 20, 2024-04-25T11:19:41.467 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T11:19:41.467 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 21.50119333 2024-04-25T11:19:41.467 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:19:41.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 -- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb76406f400 msgr2=0x7fb764071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb76406f400 0x7fb764071910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7740099b0 tx=0x7fb774008040 comp rx=0 tx=0).stop 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 -- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78415cb50 msgr2=0x7fb784156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78415cb50 0x7fb784156810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb77c01c190 tx=0x7fb770005840 comp rx=0 tx=0).stop 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 -- 172.21.15.106:0/842510010 shutdown_connections 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb76406f400 0x7fb764071910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78415cb50 0x7fb784156810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784156050 0x7fb78415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 --2- 172.21.15.106:0/842510010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7700051d0 0x7fb7841fa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 -- 172.21.15.106:0/842510010 >> 172.21.15.106:0/842510010 conn(0x7fb78406d7f0 msgr2=0x7fb7840d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 -- 172.21.15.106:0/842510010 shutdown_connections 2024-04-25T11:19:41.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.465+0000 7fb7897d8640 1 -- 172.21.15.106:0/842510010 wait complete. 2024-04-25T11:19:41.480 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T11:19:41.490 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T11:19:41.490 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T11:19:41.491 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T11:19:41.491 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T11:19:41.514 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T11:19:41.514 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T11:19:41.514 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T11:19:41.515 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:19:41.515 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:41.515 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T11:19:41.515 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:19:41.515 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T11:19:41.515 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T11:19:41.515 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:19:41.515 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:41.515 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T11:19:41.515 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:19:41.515 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T11:19:41.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:41.516 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T11:19:41.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:41.521 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-25T11:19:41.574 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2024-04-25T11:19:41.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:19:41.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:41.576 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-25T11:19:41.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:41.583 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-04-25T11:19:41.645 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2024-04-25T11:19:41.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:19:41.646 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T11:19:41.646 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:19:41.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc9824640 1 Processor -- start 2024-04-25T11:19:41.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc9824640 1 -- start start 2024-04-25T11:19:41.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc9824640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7facc40cf910 0x7facc40d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc9824640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7facc40b2e90 0x7facc40b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc9824640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40b37e0 0x7facc4161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc9824640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7facc40cfd20 con 0x7facc40b2e90 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc9824640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7facc40b2730 con 0x7facc40cf910 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc9824640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7facc40d0c70 con 0x7facc40b37e0 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc27fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7facc40b2e90 0x7facc40b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc27fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7facc40b2e90 0x7facc40b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44788/0 (socket says 172.21.15.106:44788) 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc27fc640 1 -- 172.21.15.106:0/876087641 learned_addr learned my addr 172.21.15.106:0/876087641 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc37fe640 1 --2- 172.21.15.106:0/876087641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40b37e0 0x7facc4161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc2ffd640 1 --2- 172.21.15.106:0/876087641 >> v2:172.21.15.134:3300/0 conn(0x7facc40cf910 0x7facc40d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc27fc640 1 -- 172.21.15.106:0/876087641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40b37e0 msgr2=0x7facc4161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc27fc640 1 --2- 172.21.15.106:0/876087641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40b37e0 0x7facc4161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc27fc640 1 -- 172.21.15.106:0/876087641 >> v2:172.21.15.134:3300/0 conn(0x7facc40cf910 msgr2=0x7facc40d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc27fc640 1 --2- 172.21.15.106:0/876087641 >> v2:172.21.15.134:3300/0 conn(0x7facc40cf910 0x7facc40d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.797+0000 7facc27fc640 1 -- 172.21.15.106:0/876087641 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40cf3f0 con 0x7facc40b2e90 2024-04-25T11:19:41.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc27fc640 1 --2- 172.21.15.106:0/876087641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b2e90 0x7facc40b32a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7facc40733e0 tx=0x7facb801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229d2a5d6e1c7caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc1ffb640 1 -- 172.21.15.106:0/876087641 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb80b0070 con 0x7facc40b2e90 2024-04-25T11:19:41.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc1ffb640 1 -- 172.21.15.106:0/876087641 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb80ab600 con 0x7facc40b2e90 2024-04-25T11:19:41.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc1ffb640 1 -- 172.21.15.106:0/876087641 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb80ab9d0 con 0x7facc40b2e90 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 -- 172.21.15.106:0/876087641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b2e90 msgr2=0x7facc40b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 --2- 172.21.15.106:0/876087641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b2e90 0x7facc40b32a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7facc40733e0 tx=0x7facb801c260 comp rx=0 tx=0).stop 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 -- 172.21.15.106:0/876087641 shutdown_connections 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 --2- 172.21.15.106:0/876087641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40b37e0 0x7facc4161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 --2- 172.21.15.106:0/876087641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b2e90 0x7facc40b32a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 --2- 172.21.15.106:0/876087641 >> v2:172.21.15.134:3300/0 conn(0x7facc40cf910 0x7facc40d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 -- 172.21.15.106:0/876087641 >> 172.21.15.106:0/876087641 conn(0x7facc40cffb0 msgr2=0x7facc40d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 -- 172.21.15.106:0/876087641 shutdown_connections 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.801+0000 7facc9824640 1 -- 172.21.15.106:0/876087641 wait complete. 2024-04-25T11:19:41.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 Processor -- start 2024-04-25T11:19:41.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 -- start start 2024-04-25T11:19:41.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc40b2e90 0x7facc415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b37e0 0x7facc40ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40cf910 0x7facc415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7facc40b8960 con 0x7facc40b37e0 2024-04-25T11:19:41.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7facc40b90c0 con 0x7facc40b2e90 2024-04-25T11:19:41.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7facc40ce140 con 0x7facc40cf910 2024-04-25T11:19:41.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b37e0 0x7facc40ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b37e0 0x7facc40ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44798/0 (socket says 172.21.15.106:44798) 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 -- 172.21.15.106:0/3587193403 learned_addr learned my addr 172.21.15.106:0/3587193403 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc2ffd640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc40b2e90 0x7facc415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc37fe640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40cf910 0x7facc415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 -- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40cf910 msgr2=0x7facc415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40cf910 0x7facc415c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 -- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc40b2e90 msgr2=0x7facc415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc40b2e90 0x7facc415a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 -- 172.21.15.106:0/3587193403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc415adf0 con 0x7facc40b37e0 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc2ffd640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc40b2e90 0x7facc415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc27fc640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b37e0 0x7facc40ccb20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7facb8000c10 tx=0x7facb8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facabfff640 1 -- 172.21.15.106:0/3587193403 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb80b0040 con 0x7facc40b37e0 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facabfff640 1 -- 172.21.15.106:0/3587193403 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb80b9070 con 0x7facc40b37e0 2024-04-25T11:19:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc415b320 con 0x7facc40b37e0 2024-04-25T11:19:41.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facabfff640 1 -- 172.21.15.106:0/3587193403 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb80b4610 con 0x7facc40b37e0 2024-04-25T11:19:41.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc415b8e0 con 0x7facc40b37e0 2024-04-25T11:19:41.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.805+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc4069d40 con 0x7facc40b37e0 2024-04-25T11:19:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.809+0000 7facabfff640 1 -- 172.21.15.106:0/3587193403 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7facb80b47b0 con 0x7facc40b37e0 2024-04-25T11:19:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.809+0000 7facabfff640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facb006f400 0x7facb0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.809+0000 7facc2ffd640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facb006f400 0x7facb0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.809+0000 7facabfff640 1 -- 172.21.15.106:0/3587193403 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7facb8129a50 con 0x7facc40b37e0 2024-04-25T11:19:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.809+0000 7facc2ffd640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facb006f400 0x7facb0071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7facb4006fd0 tx=0x7facb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:41.813+0000 7facabfff640 1 -- 172.21.15.106:0/3587193403 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facb8016110 con 0x7facc40b37e0 2024-04-25T11:19:42.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.005+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7facc4070880 con 0x7facc40b37e0 2024-04-25T11:19:42.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.005+0000 7facabfff640 1 -- 172.21.15.106:0/3587193403 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7facb80f6920 con 0x7facc40b37e0 2024-04-25T11:19:42.006 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facb006f400 msgr2=0x7facb0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facb006f400 0x7facb0071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7facb4006fd0 tx=0x7facb4008040 comp rx=0 tx=0).stop 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b37e0 msgr2=0x7facc40ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b37e0 0x7facc40ccb20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7facb8000c10 tx=0x7facb8019000 comp rx=0 tx=0).stop 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 shutdown_connections 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc40cf910 0x7facc415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facb006f400 0x7facb0071910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc40b37e0 0x7facc40ccb20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 --2- 172.21.15.106:0/3587193403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc40b2e90 0x7facc415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 >> 172.21.15.106:0/3587193403 conn(0x7facc40cffb0 msgr2=0x7facc409ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 shutdown_connections 2024-04-25T11:19:42.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42.009+0000 7facc9824640 1 -- 172.21.15.106:0/3587193403 wait complete. 2024-04-25T11:19:42.025 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:19:42.025 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:19:42.025 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-25T11:19:42.026 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:19:42.026 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:42.026 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:19:42.026 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:19:42.026 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:19:42.026 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:19:42.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:42.026 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-25T11:19:42.032 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:42.032 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:42.032 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:42.032 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:42.033 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:19:42.034 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42502sec preferred_lft 42502sec 2024-04-25T11:19:42.034 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-25T11:19:42.034 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:42.034 INFO:teuthology.orchestra.run.smithi106.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T11:19:42.034 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 02:42:4f:a1:63:df brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:42.034 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T11:19:42.034 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:42.034 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:19:42.035 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:42.035 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:42.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:19:42.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2024-04-25T11:19:42.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2024-04-25T11:19:42.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:19:42.035 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:42.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:42.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:42.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:42.153 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:19:42.205 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-25T11:19:42.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:42.207 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:42.275 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:19:42.276 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:42.276 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:42.276 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:19:42.276 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:19:42.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:19:42.277 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:42.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:42.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:42.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:42.392 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T11:19:42.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:42.438 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-25T11:19:42.486 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:42.486 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:42.486 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:42.487 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T11:19:42.487 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:42.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:42.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:42.599 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T11:19:42.599 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:42.599 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:42.600 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T11:19:42.600 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:19:42.600 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T11:19:42.600 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T11:19:42.600 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T11:19:42.600 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:42.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:42.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:42.698 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:42.699 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:42.699 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-25T11:19:42.699 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:19:42.699 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:42.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:42.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:42.798 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T11:19:42.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:42.798 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:42.846 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:19:42.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:42.846 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:42.893 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:19:42.950 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:42.999 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:19:42.999 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:19:42.999 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:19:42.999 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:19:42.999 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:19:42.999 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:19:43.000 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:19:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:19:43.002 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:43.002 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:43.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:43.003 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:19:43.050 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T11:19:43.050 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T11:19:43.099 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:19:43.115 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:19:43.163 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:19:43.164 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:19:43.165 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:19:43.165 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:43.165 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:43.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:43.165 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:19:43.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:2024-04-25T11:19:43.209+0000 7ff341bb2580 -1 init, newargv = 0x55fda6133b40 newargc=15 2024-04-25T11:19:43.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[35102]: starting ceph client 2024-04-25T11:19:43.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[35102]: starting fuse 2024-04-25T11:19:44.311 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:19:44.328 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:19:44.377 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:19:44.378 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:44.379 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:44.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:44.380 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:19:44.426 INFO:teuthology.orchestra.run.smithi106.stdout:45 2024-04-25T11:19:44.427 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-25T11:19:44.427 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-25T11:19:44.428 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-25T11:19:44.551 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.35102.asok 2024-04-25T11:19:44.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:44.562 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35102.asok 2024-04-25T11:19:44.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:44.563 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35102.asok status 2024-04-25T11:19:44.743 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:19:44.743 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-25T11:19:44.743 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:19:44.743 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:19:44.743 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "0", 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "35102", 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4629, 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4629 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:19:44.744 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 3021014207 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 3021014207 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.4629 192.168.0.1:0/3021014207", 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/3021014207", 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 8, 2024-04-25T11:19:44.745 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 20, 2024-04-25T11:19:44.746 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:19:44.746 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-25T11:19:44.746 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-25T11:19:44.746 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:19:44.759 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:19:44.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.908+0000 7f607dbe5640 1 Processor -- start 2024-04-25T11:19:44.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.908+0000 7f607dbe5640 1 -- start start 2024-04-25T11:19:44.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f607dbe5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6078160f10 0x7f607815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f607dbe5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60780b5cc0 con 0x7f607815d6f0 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f607dbe5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f607815cf50 con 0x7f60780b4b00 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f607dbe5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f60780b5f80 con 0x7f6078160f10 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f60777fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6078160f10 0x7f607815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f60777fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6078160f10 0x7f607815ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44802/0 (socket says 172.21.15.106:44802) 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f60777fe640 1 -- 172.21.15.106:0/1771186069 learned_addr learned my addr 172.21.15.106:0/1771186069 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f6076ffd640 1 --1- 172.21.15.106:0/1771186069 >> v1:172.21.15.106:6789/0 conn(0x7f607815d6f0 0x7f60780b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54316/0 (socket says 172.21.15.106:54316) 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f60777fe640 1 -- 172.21.15.106:0/1771186069 >> v1:172.21.15.134:6789/0 conn(0x7f60780b4b00 legacy=0x7f60780d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f60777fe640 1 -- 172.21.15.106:0/1771186069 >> v1:172.21.15.106:6789/0 conn(0x7f607815d6f0 legacy=0x7f60780b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.912+0000 7f60777fe640 1 -- 172.21.15.106:0/1771186069 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60780b6260 con 0x7f6078160f10 2024-04-25T11:19:44.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f60777fe640 1 --2- 172.21.15.106:0/1771186069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f607815ca10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6068009870 tx=0x7f606801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c534a184efe11a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f60767fc640 1 -- 172.21.15.106:0/1771186069 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60680b0070 con 0x7f6078160f10 2024-04-25T11:19:44.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f60767fc640 1 -- 172.21.15.106:0/1771186069 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60680ab600 con 0x7f6078160f10 2024-04-25T11:19:44.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f60767fc640 1 -- 172.21.15.106:0/1771186069 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60680ab9d0 con 0x7f6078160f10 2024-04-25T11:19:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 -- 172.21.15.106:0/1771186069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 msgr2=0x7f607815ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 --2- 172.21.15.106:0/1771186069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f607815ca10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6068009870 tx=0x7f606801c280 comp rx=0 tx=0).stop 2024-04-25T11:19:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 -- 172.21.15.106:0/1771186069 shutdown_connections 2024-04-25T11:19:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 --2- 172.21.15.106:0/1771186069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f607815ca10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 -- 172.21.15.106:0/1771186069 >> 172.21.15.106:0/1771186069 conn(0x7f607806c7c0 msgr2=0x7f607806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 -- 172.21.15.106:0/1771186069 shutdown_connections 2024-04-25T11:19:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 -- 172.21.15.106:0/1771186069 wait complete. 2024-04-25T11:19:44.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 Processor -- start 2024-04-25T11:19:44.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.916+0000 7f607dbe5640 1 -- start start 2024-04-25T11:19:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60780b4b00 0x7f607814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f607815d6f0 0x7f607814a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f6078072a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f607816b030 con 0x7f6078160f10 2024-04-25T11:19:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f607816aeb0 con 0x7f60780b4b00 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f607816b1b0 con 0x7f607815d6f0 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f6078072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f6078072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44814/0 (socket says 172.21.15.106:44814) 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 -- 172.21.15.106:0/2243824278 learned_addr learned my addr 172.21.15.106:0/2243824278 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f60777fe640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60780b4b00 0x7f607814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 -- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f607815d6f0 msgr2=0x7f607814a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6076ffd640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f607815d6f0 0x7f607814a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f607815d6f0 0x7f607814a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 -- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60780b4b00 msgr2=0x7f607814efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60780b4b00 0x7f607814efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 -- 172.21.15.106:0/2243824278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6078151ab0 con 0x7f6078160f10 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f60777fe640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60780b4b00 0x7f607814efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f6077fff640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f6078072a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f607000ba10 tx=0x7f607000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f605ffff640 1 -- 172.21.15.106:0/2243824278 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070019070 con 0x7f6078160f10 2024-04-25T11:19:44.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f605ffff640 1 -- 172.21.15.106:0/2243824278 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607000cb40 con 0x7f6078160f10 2024-04-25T11:19:44.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6078151d90 con 0x7f6078160f10 2024-04-25T11:19:44.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f605ffff640 1 -- 172.21.15.106:0/2243824278 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070014440 con 0x7f6078160f10 2024-04-25T11:19:44.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607814aa40 con 0x7f6078160f10 2024-04-25T11:19:44.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60781517d0 con 0x7f6078160f10 2024-04-25T11:19:44.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f605ffff640 1 -- 172.21.15.106:0/2243824278 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f607001c070 con 0x7f6078160f10 2024-04-25T11:19:44.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f605ffff640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f606406f400 0x7f6064071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f605ffff640 1 -- 172.21.15.106:0/2243824278 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f607008ef60 con 0x7f6078160f10 2024-04-25T11:19:44.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.920+0000 7f60777fe640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f606406f400 0x7f6064071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.924+0000 7f60777fe640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f606406f400 0x7f6064071910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f60680ad000 tx=0x7f6068008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:44.928+0000 7f605ffff640 1 -- 172.21.15.106:0/2243824278 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f607005bee0 con 0x7f6078160f10 2024-04-25T11:19:45.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.124+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6078154560 con 0x7f6078160f10 2024-04-25T11:19:45.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.124+0000 7f605ffff640 1 -- 172.21.15.106:0/2243824278 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f607005b7e0 con 0x7f6078160f10 2024-04-25T11:19:45.124 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:19:45.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f606406f400 msgr2=0x7f6064071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f606406f400 0x7f6064071910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f60680ad000 tx=0x7f6068008040 comp rx=0 tx=0).stop 2024-04-25T11:19:45.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 msgr2=0x7f6078072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f6078072a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f607000ba10 tx=0x7f607000bed0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 shutdown_connections 2024-04-25T11:19:45.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f606406f400 0x7f6064071910 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6078160f10 0x7f6078072a50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f607815d6f0 0x7f607814a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 --2- 172.21.15.106:0/2243824278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60780b4b00 0x7f607814efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 >> 172.21.15.106:0/2243824278 conn(0x7f607806c7c0 msgr2=0x7f607806ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:45.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 shutdown_connections 2024-04-25T11:19:45.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45.128+0000 7f607dbe5640 1 -- 172.21.15.106:0/2243824278 wait complete. 2024-04-25T11:19:45.143 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:19:45.144 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:19:45.144 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-25T11:19:45.144 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T11:19:45.144 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T11:19:45.144 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:19:45.144 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:19:45.144 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:19:45.144 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T11:19:45.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:45.144 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-25T11:19:45.150 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:45.150 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:45.150 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:45.150 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.150 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:45.150 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.150 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42499sec preferred_lft 42499sec 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.151 INFO:teuthology.orchestra.run.smithi106.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 02:42:4f:a1:63:df brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T11:19:45.152 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:45.153 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:19:45.204 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-25T11:19:45.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:45.206 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:45.260 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:19:45.261 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:45.261 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:45.261 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:19:45.261 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:19:45.261 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:19:45.261 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:45.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:45.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:45.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:45.375 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T11:19:45.422 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:19:45.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:45.423 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-25T11:19:45.470 INFO:teuthology.orchestra.run.smithi106.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T11:19:45.470 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:45.470 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:45.470 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:45.470 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T11:19:45.470 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:45.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:45.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:45.583 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T11:19:45.636 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:45.636 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether a6:8f:e0:b2:ed:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::a48f:e0ff:feb2:ed2c/64 scope link 2024-04-25T11:19:45.637 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:45.639 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T11:19:45.692 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether a6:8f:e0:b2:ed:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::a48f:e0ff:feb2:ed2c/64 scope link 2024-04-25T11:19:45.693 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:45.694 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T11:19:45.695 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:45.695 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:45.695 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T11:19:45.695 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T11:19:45.695 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T11:19:45.695 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T11:19:45.695 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T11:19:45.696 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:45.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:45.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:45.833 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:45.833 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:19:45.833 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.1 up 2024-04-25T11:19:45.833 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T11:19:45.833 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:19:45.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:45.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:45.891 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T11:19:45.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:45.891 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T11:19:45.938 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T11:19:45.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:45.938 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T11:19:45.985 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:19:46.039 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:46.087 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:19:46.087 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:19:46.087 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:19:46.087 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:19:46.087 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:19:46.087 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:19:46.087 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:19:46.087 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:19:46.088 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:46.089 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:46.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:46.090 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:19:46.138 INFO:teuthology.orchestra.run.smithi106.stdout:45 2024-04-25T11:19:46.139 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T11:19:46.139 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T11:19:46.183 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:19:46.197 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:46.246 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:19:46.246 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:19:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:19:46.248 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:46.249 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:46.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:46.250 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:19:46.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi106.stderr:2024-04-25T11:19:46.292+0000 7f7b3de92580 -1 init, newargv = 0x55f9d37fab40 newargc=15 2024-04-25T11:19:46.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi106.stderr:ceph-fuse[35247]: starting ceph client 2024-04-25T11:19:46.296 INFO:teuthology.orchestra.run.smithi106.stdout:45 2024-04-25T11:19:46.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi106.stderr:ceph-fuse[35247]: starting fuse 2024-04-25T11:19:47.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:19:47.415 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:47.464 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:19:47.464 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:19:47.464 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:19:47.465 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:47.467 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:47.467 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:47.467 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:47.467 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T11:19:47.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:47.468 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:19:47.514 INFO:teuthology.orchestra.run.smithi106.stdout:45 2024-04-25T11:19:47.514 INFO:teuthology.orchestra.run.smithi106.stdout:46 2024-04-25T11:19:47.515 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T11:19:47.515 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:19:47.516 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:19:47.517 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.2")) 2024-04-25T11:19:47.517 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-25T11:19:47.640 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.2.35247.asok 2024-04-25T11:19:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:47.651 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35247.asok 2024-04-25T11:19:47.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:47.652 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35247.asok status 2024-04-25T11:19:47.830 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:19:47.830 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-25T11:19:47.830 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:19:47.830 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:19:47.830 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "2", 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "35247", 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4644, 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-25T11:19:47.831 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4644 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.2:0", 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 212241860 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.2:0", 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 212241860 2024-04-25T11:19:47.832 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.4644 192.168.0.2:0/212241860", 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.2:0/212241860", 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 8, 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 20, 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-25T11:19:47.833 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:19:47.846 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:19:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:47.996+0000 7f759001e640 1 Processor -- start 2024-04-25T11:19:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f759001e640 1 -- start start 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f759001e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f75880d2160 0x7f75880d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f759001e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75880b5120 0x7f75880b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f759001e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75880d2ab0 con 0x7f75880ba970 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f759001e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f75880d1100 con 0x7f75880b5120 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f759001e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f75880d0530 con 0x7f75880d2160 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f758e5bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75880b5120 0x7f75880b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f758e5bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75880b5120 0x7f75880b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39406/0 (socket says 172.21.15.106:39406) 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f758e5bb640 1 -- 172.21.15.106:0/3486041263 learned_addr learned my addr 172.21.15.106:0/3486041263 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:48.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f758d5b9640 1 --2- 172.21.15.106:0/3486041263 >> v2:172.21.15.134:3300/0 conn(0x7f75880d2160 0x7f75880d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f758e5bb640 1 -- 172.21.15.106:0/3486041263 >> v2:172.21.15.134:3300/0 conn(0x7f75880d2160 msgr2=0x7f75880d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:48.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f758e5bb640 1 --2- 172.21.15.106:0/3486041263 >> v2:172.21.15.134:3300/0 conn(0x7f75880d2160 0x7f75880d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f758e5bb640 1 -- 172.21.15.106:0/3486041263 >> v1:172.21.15.134:6789/0 conn(0x7f75880ba970 legacy=0x7f75880d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:48.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.000+0000 7f758e5bb640 1 -- 172.21.15.106:0/3486041263 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75880d07d0 con 0x7f75880b5120 2024-04-25T11:19:48.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f758e5bb640 1 --2- 172.21.15.106:0/3486041263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f75880b5530 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75880d68c0 tx=0x7f758001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa15ac5941781861 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:48.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f758cdb8640 1 -- 172.21.15.106:0/3486041263 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800ac020 con 0x7f75880b5120 2024-04-25T11:19:48.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f758cdb8640 1 -- 172.21.15.106:0/3486041263 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75800a6410 con 0x7f75880b5120 2024-04-25T11:19:48.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f758cdb8640 1 -- 172.21.15.106:0/3486041263 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800a67e0 con 0x7f75880b5120 2024-04-25T11:19:48.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f759001e640 1 -- 172.21.15.106:0/3486041263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 msgr2=0x7f75880b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f759001e640 1 --2- 172.21.15.106:0/3486041263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f75880b5530 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75880d68c0 tx=0x7f758001c740 comp rx=0 tx=0).stop 2024-04-25T11:19:48.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f759001e640 1 -- 172.21.15.106:0/3486041263 shutdown_connections 2024-04-25T11:19:48.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f759001e640 1 --2- 172.21.15.106:0/3486041263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f75880b5530 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f759001e640 1 --2- 172.21.15.106:0/3486041263 >> v2:172.21.15.134:3300/0 conn(0x7f75880d2160 0x7f75880d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f759001e640 1 -- 172.21.15.106:0/3486041263 >> 172.21.15.106:0/3486041263 conn(0x7f75880d57e0 msgr2=0x7f75880d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:48.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f759001e640 1 -- 172.21.15.106:0/3486041263 shutdown_connections 2024-04-25T11:19:48.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.004+0000 7f759001e640 1 -- 172.21.15.106:0/3486041263 wait complete. 2024-04-25T11:19:48.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 Processor -- start 2024-04-25T11:19:48.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 -- start start 2024-04-25T11:19:48.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f758815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75880ba970 0x7f75881fd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75880d2160 0x7f75880b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75880bb200 con 0x7f75880b5120 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75880bb080 con 0x7f75880d2160 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f75880b5a70 con 0x7f75880ba970 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f758815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f758815a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39412/0 (socket says 172.21.15.106:39412) 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 -- 172.21.15.106:0/960955118 learned_addr learned my addr 172.21.15.106:0/960955118 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758e5bb640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75880d2160 0x7f75880b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758d5b9640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75880ba970 0x7f75881fd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 -- 172.21.15.106:0/960955118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75880ba970 msgr2=0x7f75881fd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75880ba970 0x7f75881fd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 -- 172.21.15.106:0/960955118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75880d2160 msgr2=0x7f75880b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75880d2160 0x7f75880b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 -- 172.21.15.106:0/960955118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75881fe280 con 0x7f75880b5120 2024-04-25T11:19:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f758ddba640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f758815a7e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f758400b3e0 tx=0x7f758400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f757e7fc640 1 -- 172.21.15.106:0/960955118 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584015030 con 0x7f75880b5120 2024-04-25T11:19:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f757e7fc640 1 -- 172.21.15.106:0/960955118 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7584005970 con 0x7f75880b5120 2024-04-25T11:19:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f757e7fc640 1 -- 172.21.15.106:0/960955118 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584005d60 con 0x7f75880b5120 2024-04-25T11:19:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75881fa220 con 0x7f75880b5120 2024-04-25T11:19:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75881fa7e0 con 0x7f75880b5120 2024-04-25T11:19:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.008+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75880d3630 con 0x7f75880b5120 2024-04-25T11:19:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.012+0000 7f757e7fc640 1 -- 172.21.15.106:0/960955118 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7584018070 con 0x7f75880b5120 2024-04-25T11:19:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.012+0000 7f757e7fc640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756c06f430 0x7f756c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.012+0000 7f757e7fc640 1 -- 172.21.15.106:0/960955118 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f758408f390 con 0x7f75880b5120 2024-04-25T11:19:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.012+0000 7f758d5b9640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756c06f430 0x7f756c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.012+0000 7f758d5b9640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756c06f430 0x7f756c071940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7578006fd0 tx=0x7f7578008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:48.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.016+0000 7f757e7fc640 1 -- 172.21.15.106:0/960955118 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f758405c360 con 0x7f75880b5120 2024-04-25T11:19:48.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f75880b4990 con 0x7f75880b5120 2024-04-25T11:19:48.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f757e7fc640 1 -- 172.21.15.106:0/960955118 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f75840643e0 con 0x7f75880b5120 2024-04-25T11:19:48.219 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:19:48.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756c06f430 msgr2=0x7f756c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756c06f430 0x7f756c071940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7578006fd0 tx=0x7f7578008040 comp rx=0 tx=0).stop 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 msgr2=0x7f758815a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f758815a7e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f758400b3e0 tx=0x7f758400b8a0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 shutdown_connections 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75880d2160 0x7f75880b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75880ba970 0x7f75881fd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756c06f430 0x7f756c071940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 --2- 172.21.15.106:0/960955118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75880b5120 0x7f758815a7e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 >> 172.21.15.106:0/960955118 conn(0x7f75880d57e0 msgr2=0x7f758815cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 shutdown_connections 2024-04-25T11:19:48.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:48.220+0000 7f759001e640 1 -- 172.21.15.106:0/960955118 wait complete. 2024-04-25T11:19:48.239 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:19:48.239 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:19:48.239 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-04-25T11:19:48.239 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:19:48.239 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:48.239 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:19:48.239 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:19:48.239 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:19:48.240 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:19:48.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:48.240 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-04-25T11:19:48.247 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:48.247 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:48.247 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:48.247 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:48.247 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:48.247 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:48.248 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:19:48.249 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42494sec preferred_lft 42494sec 2024-04-25T11:19:48.249 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::ec4:7aff:fe88:6de9/64 scope link 2024-04-25T11:19:48.249 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:48.249 INFO:teuthology.orchestra.run.smithi134.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T11:19:48.249 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 02:42:49:ad:42:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:48.249 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T11:19:48.249 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:48.249 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:19:48.249 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:48.249 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:48.249 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:19:48.250 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr flush dev ceph-brx 2024-04-25T11:19:48.250 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set ceph-brx up 2024-04-25T11:19:48.250 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:19:48.250 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:48.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:48.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:48.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:48.379 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:19:48.388 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-25T11:19:48.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:48.390 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-04-25T11:19:48.455 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:48.456 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:19:48.456 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:48.457 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:48.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:19:48.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:19:48.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:19:48.457 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:48.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:48.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:48.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:48.577 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-25T11:19:48.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:48.625 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list-id 2024-04-25T11:19:48.672 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:48.672 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:48.673 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:48.673 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T11:19:48.673 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:48.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:48.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:48.792 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T11:19:48.792 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:48.792 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:48.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T11:19:48.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:19:48.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T11:19:48.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T11:19:48.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T11:19:48.792 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:48.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:48.900 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:48.900 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:48.900 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.0 up 2024-04-25T11:19:48.900 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:19:48.900 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:48.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:48.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:49.004 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T11:19:49.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:49.004 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:49.052 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:19:49.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:49.052 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:49.099 DEBUG:teuthology.orchestra.run.smithi134:> sudo modprobe fuse 2024-04-25T11:19:49.156 DEBUG:teuthology.orchestra.run.smithi134:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/sys 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/proc 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/dev 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/pts 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/run 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/ 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/security 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/shm 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/run/lock 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/pstore 2024-04-25T11:19:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/bpf 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/hugepages 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/mqueue 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/debug 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/tracing 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/fs/nfsd 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/config 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/run/rpc_pipefs 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/run/user/1000 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T11:19:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T11:19:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T11:19:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns 2024-04-25T11:19:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:49.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:49.214 DEBUG:teuthology.orchestra.run.smithi134:> ls /sys/fs/fuse/connections 2024-04-25T11:19:49.261 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T11:19:49.261 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T11:19:49.305 DEBUG:teuthology.orchestra.run.smithi134:> sudo modprobe fuse 2024-04-25T11:19:49.320 DEBUG:teuthology.orchestra.run.smithi134:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:49.372 INFO:teuthology.orchestra.run.smithi134.stdout:/sys 2024-04-25T11:19:49.372 INFO:teuthology.orchestra.run.smithi134.stdout:/proc 2024-04-25T11:19:49.372 INFO:teuthology.orchestra.run.smithi134.stdout:/dev 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/pts 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/run 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/ 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/security 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/shm 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/run/lock 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/pstore 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/bpf 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/hugepages 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/mqueue 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/debug 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/tracing 2024-04-25T11:19:49.373 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/fs/nfsd 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/config 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/run/rpc_pipefs 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/run/user/1000 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:49.374 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:49.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:49.375 DEBUG:teuthology.orchestra.run.smithi134:> ls /sys/fs/fuse/connections 2024-04-25T11:19:49.417 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi134.stderr:2024-04-25T11:19:49.418+0000 7f8525869580 -1 init, newargv = 0x7f8514001090 newargc=15 2024-04-25T11:19:49.417 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi134.stderr:ceph-fuse[33523]: starting ceph client 2024-04-25T11:19:49.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi134.stderr:ceph-fuse[33523]: starting fuse 2024-04-25T11:19:50.524 DEBUG:teuthology.orchestra.run.smithi134:> sudo modprobe fuse 2024-04-25T11:19:50.539 DEBUG:teuthology.orchestra.run.smithi134:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:50.593 INFO:teuthology.orchestra.run.smithi134.stdout:/sys 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/proc 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/dev 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/pts 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/run 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/ 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/security 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/shm 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/run/lock 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/pstore 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/bpf 2024-04-25T11:19:50.594 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/hugepages 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/mqueue 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/debug 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/tracing 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/fs/nfsd 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/config 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/run/rpc_pipefs 2024-04-25T11:19:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:/run/user/1000 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:50.596 INFO:teuthology.orchestra.run.smithi134.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:50.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:50.597 DEBUG:teuthology.orchestra.run.smithi134:> ls /sys/fs/fuse/connections 2024-04-25T11:19:50.648 INFO:teuthology.orchestra.run.smithi134.stdout:45 2024-04-25T11:19:50.649 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T11:19:50.649 DEBUG:teuthology.orchestra.run.smithi134:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:19:50.649 DEBUG:teuthology.orchestra.run.smithi134:> import glob 2024-04-25T11:19:50.649 DEBUG:teuthology.orchestra.run.smithi134:> import re 2024-04-25T11:19:50.649 DEBUG:teuthology.orchestra.run.smithi134:> import os 2024-04-25T11:19:50.649 DEBUG:teuthology.orchestra.run.smithi134:> import subprocess 2024-04-25T11:19:50.649 DEBUG:teuthology.orchestra.run.smithi134:> 2024-04-25T11:19:50.649 DEBUG:teuthology.orchestra.run.smithi134:> def _find_admin_socket(client_name): 2024-04-25T11:19:50.649 DEBUG:teuthology.orchestra.run.smithi134:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> files = glob.glob(asok_path) 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> # Given a non-glob path, it better be there 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> if "*" not in asok_path: 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> assert(len(files) == 1) 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> return files[0] 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> for f in files: 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> contents = proc_f.read() 2024-04-25T11:19:50.650 DEBUG:teuthology.orchestra.run.smithi134:> if mountpoint in contents: 2024-04-25T11:19:50.651 DEBUG:teuthology.orchestra.run.smithi134:> return f 2024-04-25T11:19:50.651 DEBUG:teuthology.orchestra.run.smithi134:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:19:50.651 DEBUG:teuthology.orchestra.run.smithi134:> 2024-04-25T11:19:50.651 DEBUG:teuthology.orchestra.run.smithi134:> print(_find_admin_socket("client.1")) 2024-04-25T11:19:50.651 DEBUG:teuthology.orchestra.run.smithi134:> ' 2024-04-25T11:19:50.774 INFO:teuthology.orchestra.run.smithi134.stdout:/var/run/ceph/ceph-client.1.33523.asok 2024-04-25T11:19:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:50.785 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33523.asok 2024-04-25T11:19:50.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:50.786 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33523.asok status 2024-04-25T11:19:50.961 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-25T11:19:50.961 INFO:teuthology.orchestra.run.smithi134.stdout: "metadata": { 2024-04-25T11:19:50.961 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:19:50.961 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:19:50.961 INFO:teuthology.orchestra.run.smithi134.stdout: "entity_id": "1", 2024-04-25T11:19:50.961 INFO:teuthology.orchestra.run.smithi134.stdout: "hostname": "smithi134", 2024-04-25T11:19:50.961 INFO:teuthology.orchestra.run.smithi134.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T11:19:50.961 INFO:teuthology.orchestra.run.smithi134.stdout: "pid": "33523", 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "root": "/" 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "dentry_count": 0, 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "dentry_pinned_count": 0, 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "id": 4573, 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "inst": { 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "name": { 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "client", 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4573 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": { 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:19:50.962 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 3114108355 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": { 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 3114108355 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inst_str": "client.4573 192.168.0.1:0/3114108355", 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "addr_str": "192.168.0.1:0/3114108355", 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inode_count": 1, 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_epoch": 8, 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_epoch": 20, 2024-04-25T11:19:50.963 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:19:50.964 INFO:teuthology.orchestra.run.smithi134.stdout: "blocklisted": false, 2024-04-25T11:19:50.964 INFO:teuthology.orchestra.run.smithi134.stdout: "fs_name": "cephfs" 2024-04-25T11:19:50.964 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-25T11:19:50.977 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:19:51.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.136+0000 7fd08f3f8640 1 Processor -- start 2024-04-25T11:19:51.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.136+0000 7fd08f3f8640 1 -- start start 2024-04-25T11:19:51.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08f3f8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0880b43f0 0x7fd088072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08f3f8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd088072f90 0x7fd088160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08f3f8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0880733a0 con 0x7fd08815c270 2024-04-25T11:19:51.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08f3f8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd08815fdb0 con 0x7fd0880b43f0 2024-04-25T11:19:51.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08f3f8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd0880b49b0 con 0x7fd088072f90 2024-04-25T11:19:51.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08d995640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd088072f90 0x7fd088160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08c993640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0880b43f0 0x7fd088072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08c993640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0880b43f0 0x7fd088072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:33904/0 (socket says 172.21.15.106:33904) 2024-04-25T11:19:51.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08d995640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd088072f90 0x7fd088160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:39900/0 (socket says 172.21.15.106:39900) 2024-04-25T11:19:51.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08c993640 1 -- 172.21.15.106:0/917166914 learned_addr learned my addr 172.21.15.106:0/917166914 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:51.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08d194640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd08815c270 0x7fd0880cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:40004/0 (socket says 172.21.15.106:40004) 2024-04-25T11:19:51.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08c993640 1 -- 172.21.15.106:0/917166914 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd088072f90 msgr2=0x7fd088160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:51.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08c993640 1 --2- 172.21.15.106:0/917166914 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd088072f90 0x7fd088160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08c993640 1 -- 172.21.15.106:0/917166914 >> v1:172.21.15.134:6789/0 conn(0x7fd08815c270 legacy=0x7fd0880cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.140+0000 7fd08c993640 1 -- 172.21.15.106:0/917166914 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0880b4c90 con 0x7fd0880b43f0 2024-04-25T11:19:51.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08c993640 1 --2- 172.21.15.106:0/917166914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 0x7fd088072a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd078009870 tx=0x7fd07801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f65bb122222968aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:51.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd07ffff640 1 -- 172.21.15.106:0/917166914 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0780b0070 con 0x7fd0880b43f0 2024-04-25T11:19:51.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd07ffff640 1 -- 172.21.15.106:0/917166914 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0780ab600 con 0x7fd0880b43f0 2024-04-25T11:19:51.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd07ffff640 1 -- 172.21.15.106:0/917166914 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0780ab9d0 con 0x7fd0880b43f0 2024-04-25T11:19:51.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 -- 172.21.15.106:0/917166914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 msgr2=0x7fd088072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/917166914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 0x7fd088072a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd078009870 tx=0x7fd07801c560 comp rx=0 tx=0).stop 2024-04-25T11:19:51.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 -- 172.21.15.106:0/917166914 shutdown_connections 2024-04-25T11:19:51.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/917166914 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd088072f90 0x7fd088160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/917166914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 0x7fd088072a50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 -- 172.21.15.106:0/917166914 >> 172.21.15.106:0/917166914 conn(0x7fd08806c7c0 msgr2=0x7fd08806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:51.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 -- 172.21.15.106:0/917166914 shutdown_connections 2024-04-25T11:19:51.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 -- 172.21.15.106:0/917166914 wait complete. 2024-04-25T11:19:51.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 Processor -- start 2024-04-25T11:19:51.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.144+0000 7fd08f3f8640 1 -- start start 2024-04-25T11:19:51.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd088072f90 0x7fd0880b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 0x7fd088167e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd08815c270 0x7fd0881683c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd0880cf3d0 con 0x7fd088072f90 2024-04-25T11:19:51.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd088073c00 con 0x7fd0880b43f0 2024-04-25T11:19:51.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd088166e70 con 0x7fd08815c270 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd088072f90 0x7fd0880b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd088072f90 0x7fd0880b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39428/0 (socket says 172.21.15.106:39428) 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 -- 172.21.15.106:0/900489028 learned_addr learned my addr 172.21.15.106:0/900489028 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08c993640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 0x7fd088167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d995640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd08815c270 0x7fd0881683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 -- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd08815c270 msgr2=0x7fd0881683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd08815c270 0x7fd0881683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 -- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 msgr2=0x7fd088167e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 0x7fd088167e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 -- 172.21.15.106:0/900489028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08815e460 con 0x7fd088072f90 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08c993640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 0x7fd088167e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08d194640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd088072f90 0x7fd0880b1a70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c3a0 tx=0x7fd08400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:51.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd07dffb640 1 -- 172.21.15.106:0/900489028 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0840074e0 con 0x7fd088072f90 2024-04-25T11:19:51.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08815e740 con 0x7fd088072f90 2024-04-25T11:19:51.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd07dffb640 1 -- 172.21.15.106:0/900489028 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd084007680 con 0x7fd088072f90 2024-04-25T11:19:51.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd07dffb640 1 -- 172.21.15.106:0/900489028 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084004a30 con 0x7fd088072f90 2024-04-25T11:19:51.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08815ed00 con 0x7fd088072f90 2024-04-25T11:19:51.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.148+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd088166b50 con 0x7fd088072f90 2024-04-25T11:19:51.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.152+0000 7fd07dffb640 1 -- 172.21.15.106:0/900489028 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd08401a020 con 0x7fd088072f90 2024-04-25T11:19:51.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.152+0000 7fd07dffb640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd06c06f400 0x7fd06c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.152+0000 7fd08c993640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd06c06f400 0x7fd06c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.152+0000 7fd07dffb640 1 -- 172.21.15.106:0/900489028 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd08408f280 con 0x7fd088072f90 2024-04-25T11:19:51.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.152+0000 7fd08c993640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd06c06f400 0x7fd06c071910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd0780ad000 tx=0x7fd078008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:51.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.156+0000 7fd07dffb640 1 -- 172.21.15.106:0/900489028 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd084021070 con 0x7fd088072f90 2024-04-25T11:19:51.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd08806da30 con 0x7fd088072f90 2024-04-25T11:19:51.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd07dffb640 1 -- 172.21.15.106:0/900489028 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fd0880733a0 con 0x7fd088072f90 2024-04-25T11:19:51.347 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:19:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd06c06f400 msgr2=0x7fd06c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd06c06f400 0x7fd06c071910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd0780ad000 tx=0x7fd078008040 comp rx=0 tx=0).stop 2024-04-25T11:19:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd088072f90 msgr2=0x7fd0880b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd088072f90 0x7fd0880b1a70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c3a0 tx=0x7fd08400c860 comp rx=0 tx=0).stop 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 shutdown_connections 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd08815c270 0x7fd0881683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b43f0 0x7fd088167e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd06c06f400 0x7fd06c071910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 --2- 172.21.15.106:0/900489028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd088072f90 0x7fd0880b1a70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 >> 172.21.15.106:0/900489028 conn(0x7fd08806c7c0 msgr2=0x7fd0880cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 shutdown_connections 2024-04-25T11:19:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:51.348+0000 7fd08f3f8640 1 -- 172.21.15.106:0/900489028 wait complete. 2024-04-25T11:19:51.366 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:19:51.366 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:19:51.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-04-25T11:19:51.366 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T11:19:51.367 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T11:19:51.367 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:19:51.367 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:19:51.367 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:19:51.367 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T11:19:51.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:51.367 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:51.375 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-04-25T11:19:51.376 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::ec4:7aff:fe88:6de9/64 scope link 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 02:42:49:ad:42:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:51.377 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T11:19:51.378 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:51.378 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:19:51.427 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-25T11:19:51.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:51.429 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-04-25T11:19:51.481 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-04-25T11:19:51.481 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:19:51.481 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T11:19:51.481 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:19:51.481 INFO:teuthology.orchestra.run.smithi134.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T11:19:51.481 INFO:teuthology.orchestra.run.smithi134.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T11:19:51.481 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:19:51.481 INFO:teuthology.orchestra.run.smithi134.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:51.482 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:51.482 INFO:teuthology.orchestra.run.smithi134.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:19:51.482 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:19:51.482 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:51.482 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:51.482 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:19:51.482 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:19:51.482 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:19:51.482 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:51.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:51.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:51.605 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-25T11:19:51.611 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:19:51.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:51.612 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list-id 2024-04-25T11:19:51.656 INFO:teuthology.orchestra.run.smithi134.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T11:19:51.656 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:51.656 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:51.656 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:51.657 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T11:19:51.657 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:51.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:51.777 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T11:19:51.789 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:51.789 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:51.789 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:51.789 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.789 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:51.790 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.790 INFO:teuthology.orchestra.run.smithi134.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:51.790 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 2e:aa:4c:7b:48:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:19:51.790 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:19:51.790 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.790 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::2caa:4cff:fe7b:48e9/64 scope link 2024-04-25T11:19:51.790 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:51.791 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T11:19:51.843 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:51.843 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 2e:aa:4c:7b:48:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::2caa:4cff:fe7b:48e9/64 scope link 2024-04-25T11:19:51.844 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:51.845 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T11:19:51.845 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:51.846 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:51.846 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T11:19:51.846 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T11:19:51.846 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T11:19:51.846 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T11:19:51.846 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T11:19:51.846 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:51.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:51.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:51.981 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:51.981 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-04-25T11:19:51.981 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.1 up 2024-04-25T11:19:51.981 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T11:19:51.981 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-04-25T11:19:52.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:52.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:52.040 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T11:19:52.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:52.041 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T11:19:52.088 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T11:19:52.088 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:52.088 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T11:19:52.135 DEBUG:teuthology.orchestra.run.smithi134:> sudo modprobe fuse 2024-04-25T11:19:52.193 DEBUG:teuthology.orchestra.run.smithi134:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:52.241 INFO:teuthology.orchestra.run.smithi134.stdout:/sys 2024-04-25T11:19:52.241 INFO:teuthology.orchestra.run.smithi134.stdout:/proc 2024-04-25T11:19:52.241 INFO:teuthology.orchestra.run.smithi134.stdout:/dev 2024-04-25T11:19:52.241 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/pts 2024-04-25T11:19:52.241 INFO:teuthology.orchestra.run.smithi134.stdout:/run 2024-04-25T11:19:52.241 INFO:teuthology.orchestra.run.smithi134.stdout:/ 2024-04-25T11:19:52.241 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/security 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/shm 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/run/lock 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/pstore 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/bpf 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/hugepages 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/mqueue 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/debug 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/tracing 2024-04-25T11:19:52.242 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/fs/nfsd 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/config 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/run/rpc_pipefs 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/run/user/1000 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:52.243 INFO:teuthology.orchestra.run.smithi134.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:52.244 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:52.244 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:52.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:52.245 DEBUG:teuthology.orchestra.run.smithi134:> ls /sys/fs/fuse/connections 2024-04-25T11:19:52.292 INFO:teuthology.orchestra.run.smithi134.stdout:45 2024-04-25T11:19:52.292 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T11:19:52.293 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T11:19:52.337 DEBUG:teuthology.orchestra.run.smithi134:> sudo modprobe fuse 2024-04-25T11:19:52.351 DEBUG:teuthology.orchestra.run.smithi134:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:52.398 INFO:teuthology.orchestra.run.smithi134.stdout:/sys 2024-04-25T11:19:52.398 INFO:teuthology.orchestra.run.smithi134.stdout:/proc 2024-04-25T11:19:52.398 INFO:teuthology.orchestra.run.smithi134.stdout:/dev 2024-04-25T11:19:52.398 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/pts 2024-04-25T11:19:52.398 INFO:teuthology.orchestra.run.smithi134.stdout:/run 2024-04-25T11:19:52.399 INFO:teuthology.orchestra.run.smithi134.stdout:/ 2024-04-25T11:19:52.399 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/security 2024-04-25T11:19:52.399 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/shm 2024-04-25T11:19:52.399 INFO:teuthology.orchestra.run.smithi134.stdout:/run/lock 2024-04-25T11:19:52.399 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup 2024-04-25T11:19:52.399 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/pstore 2024-04-25T11:19:52.400 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/bpf 2024-04-25T11:19:52.400 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:52.400 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/hugepages 2024-04-25T11:19:52.400 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/mqueue 2024-04-25T11:19:52.400 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/debug 2024-04-25T11:19:52.400 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/tracing 2024-04-25T11:19:52.401 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/fs/nfsd 2024-04-25T11:19:52.401 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:52.401 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/config 2024-04-25T11:19:52.401 INFO:teuthology.orchestra.run.smithi134.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:52.401 INFO:teuthology.orchestra.run.smithi134.stdout:/run/rpc_pipefs 2024-04-25T11:19:52.401 INFO:teuthology.orchestra.run.smithi134.stdout:/run/user/1000 2024-04-25T11:19:52.402 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph 2024-04-25T11:19:52.402 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T11:19:52.402 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T11:19:52.402 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T11:19:52.402 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T11:19:52.402 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns 2024-04-25T11:19:52.403 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:52.403 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:52.403 INFO:teuthology.orchestra.run.smithi134.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:52.403 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:52.403 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:52.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:52.404 DEBUG:teuthology.orchestra.run.smithi134:> ls /sys/fs/fuse/connections 2024-04-25T11:19:52.448 INFO:teuthology.orchestra.run.smithi134.stdout:45 2024-04-25T11:19:52.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi134.stderr:2024-04-25T11:19:52.450+0000 7f5002e2b580 -1 init, newargv = 0x7f4ff0001090 newargc=15 2024-04-25T11:19:52.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi134.stderr:ceph-fuse[33642]: starting ceph client 2024-04-25T11:19:52.454 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi134.stderr:ceph-fuse[33642]: starting fuse 2024-04-25T11:19:53.550 DEBUG:teuthology.orchestra.run.smithi134:> sudo modprobe fuse 2024-04-25T11:19:53.566 DEBUG:teuthology.orchestra.run.smithi134:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/sys 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/proc 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/dev 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/pts 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/run 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/ 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/security 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/shm 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/run/lock 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup 2024-04-25T11:19:53.614 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/pstore 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/bpf 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/hugepages 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/mqueue 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/debug 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/tracing 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/fs/nfsd 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/fuse/connections 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/config 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/run/rpc_pipefs 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/run/user/1000 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T11:19:53.615 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:53.616 INFO:teuthology.orchestra.run.smithi134.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T11:19:53.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:53.617 DEBUG:teuthology.orchestra.run.smithi134:> ls /sys/fs/fuse/connections 2024-04-25T11:19:53.668 INFO:teuthology.orchestra.run.smithi134.stdout:45 2024-04-25T11:19:53.669 INFO:teuthology.orchestra.run.smithi134.stdout:46 2024-04-25T11:19:53.669 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T11:19:53.669 DEBUG:teuthology.orchestra.run.smithi134:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:19:53.669 DEBUG:teuthology.orchestra.run.smithi134:> import glob 2024-04-25T11:19:53.669 DEBUG:teuthology.orchestra.run.smithi134:> import re 2024-04-25T11:19:53.669 DEBUG:teuthology.orchestra.run.smithi134:> import os 2024-04-25T11:19:53.669 DEBUG:teuthology.orchestra.run.smithi134:> import subprocess 2024-04-25T11:19:53.669 DEBUG:teuthology.orchestra.run.smithi134:> 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> def _find_admin_socket(client_name): 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> files = glob.glob(asok_path) 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> # Given a non-glob path, it better be there 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> if "*" not in asok_path: 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> assert(len(files) == 1) 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> return files[0] 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> for f in files: 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:19:53.670 DEBUG:teuthology.orchestra.run.smithi134:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:19:53.671 DEBUG:teuthology.orchestra.run.smithi134:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:19:53.671 DEBUG:teuthology.orchestra.run.smithi134:> contents = proc_f.read() 2024-04-25T11:19:53.671 DEBUG:teuthology.orchestra.run.smithi134:> if mountpoint in contents: 2024-04-25T11:19:53.671 DEBUG:teuthology.orchestra.run.smithi134:> return f 2024-04-25T11:19:53.671 DEBUG:teuthology.orchestra.run.smithi134:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:19:53.671 DEBUG:teuthology.orchestra.run.smithi134:> 2024-04-25T11:19:53.671 DEBUG:teuthology.orchestra.run.smithi134:> print(_find_admin_socket("client.3")) 2024-04-25T11:19:53.671 DEBUG:teuthology.orchestra.run.smithi134:> ' 2024-04-25T11:19:53.791 INFO:teuthology.orchestra.run.smithi134.stdout:/var/run/ceph/ceph-client.3.33642.asok 2024-04-25T11:19:53.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T11:19:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:53.802 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33642.asok 2024-04-25T11:19:53.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:53.803 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33642.asok status 2024-04-25T11:19:53.982 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-25T11:19:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: "metadata": { 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "entity_id": "3", 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "hostname": "smithi134", 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "pid": "33642", 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "root": "/" 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "dentry_count": 0, 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "dentry_pinned_count": 0, 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "id": 4469, 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "inst": { 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "name": { 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "client", 2024-04-25T11:19:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4469 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": { 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "192.168.0.2:0", 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 14215116 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": { 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "192.168.0.2:0", 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 14215116 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "inst_str": "client.4469 192.168.0.2:0/14215116", 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr_str": "192.168.0.2:0/14215116", 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "inode_count": 1, 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_epoch": 8, 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_epoch": 20, 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "blocklisted": false, 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "fs_name": "cephfs" 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-25T11:19:53.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:53.999 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-25T11:19:53.999 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:54.006 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-25T11:19:54.006 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:54.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:54.007 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:54.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:54.066 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-25T11:19:54.067 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T11:19:54.116 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-25T11:19:54.116 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T11:19:54.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:54.116 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T11:19:54.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:54.177 DEBUG:teuthology.orchestra.run.smithi134:> stat --file-system '--printf=%T 2024-04-25T11:19:54.177 DEBUG:teuthology.orchestra.run.smithi134:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:54.184 INFO:teuthology.orchestra.run.smithi134.stdout:fuseblk 2024-04-25T11:19:54.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:54.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:54.184 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:54.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:54.241 DEBUG:teuthology.orchestra.run.smithi134:> stat --file-system '--printf=%T 2024-04-25T11:19:54.241 DEBUG:teuthology.orchestra.run.smithi134:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T11:19:54.289 INFO:teuthology.orchestra.run.smithi134.stdout:fuseblk 2024-04-25T11:19:54.289 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T11:19:54.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:54.290 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T11:19:54.349 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T11:19:54.364 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:54.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e73f75640 1 Processor -- start 2024-04-25T11:19:54.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e73f75640 1 -- start start 2024-04-25T11:19:54.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e73f75640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c15e0a0 0x7f2e6c073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e73f75640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e6c0d0ce0 0x7f2e6c0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e73f75640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e6c0ce050 con 0x7f2e6c0ce850 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e73f75640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6c15e6b0 con 0x7f2e6c0d0ce0 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e73f75640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c15e990 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e72512640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e6c0d0ce0 0x7f2e6c0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e72512640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e6c0d0ce0 0x7f2e6c0d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39438/0 (socket says 172.21.15.106:39438) 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e72512640 1 -- 172.21.15.106:0/3534026022 learned_addr learned my addr 172.21.15.106:0/3534026022 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e71d11640 1 --2- 172.21.15.106:0/3534026022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c15e0a0 0x7f2e6c073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e72512640 1 -- 172.21.15.106:0/3534026022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c15e0a0 msgr2=0x7f2e6c073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e72512640 1 --2- 172.21.15.106:0/3534026022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c15e0a0 0x7f2e6c073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e72512640 1 -- 172.21.15.106:0/3534026022 >> v1:172.21.15.134:6789/0 conn(0x7f2e6c0ce850 legacy=0x7f2e6c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.528+0000 7f2e72512640 1 -- 172.21.15.106:0/3534026022 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c15ec70 con 0x7f2e6c0d0ce0 2024-04-25T11:19:54.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e72512640 1 --2- 172.21.15.106:0/3534026022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c0d0ce0 0x7f2e6c0d10f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009e50 tx=0x7f2e6401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad963bc78e071d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e70d0f640 1 -- 172.21.15.106:0/3534026022 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640a56c0 con 0x7f2e6c0d0ce0 2024-04-25T11:19:54.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e70d0f640 1 -- 172.21.15.106:0/3534026022 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e640a5860 con 0x7f2e6c0d0ce0 2024-04-25T11:19:54.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e70d0f640 1 -- 172.21.15.106:0/3534026022 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640a5c30 con 0x7f2e6c0d0ce0 2024-04-25T11:19:54.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e73f75640 1 -- 172.21.15.106:0/3534026022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c0d0ce0 msgr2=0x7f2e6c0d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e73f75640 1 --2- 172.21.15.106:0/3534026022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c0d0ce0 0x7f2e6c0d10f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009e50 tx=0x7f2e6401c9f0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e73f75640 1 -- 172.21.15.106:0/3534026022 shutdown_connections 2024-04-25T11:19:54.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e73f75640 1 --2- 172.21.15.106:0/3534026022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c0d0ce0 0x7f2e6c0d10f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e73f75640 1 --2- 172.21.15.106:0/3534026022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c15e0a0 0x7f2e6c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e73f75640 1 -- 172.21.15.106:0/3534026022 >> 172.21.15.106:0/3534026022 conn(0x7f2e6c06c7c0 msgr2=0x7f2e6c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:54.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e73f75640 1 -- 172.21.15.106:0/3534026022 shutdown_connections 2024-04-25T11:19:54.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.532+0000 7f2e73f75640 1 -- 172.21.15.106:0/3534026022 wait complete. 2024-04-25T11:19:54.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 Processor -- start 2024-04-25T11:19:54.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 -- start start 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c0ce850 0x7f2e6c141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e6c0d0ce0 0x7f2e6c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c15e0a0 0x7f2e6c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c16a870 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c16a6f0 con 0x7f2e6c0d0ce0 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c16a9f0 con 0x7f2e6c0ce850 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c15e0a0 0x7f2e6c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c15e0a0 0x7f2e6c15bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39454/0 (socket says 172.21.15.106:39454) 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 -- 172.21.15.106:0/4196395677 learned_addr learned my addr 172.21.15.106:0/4196395677 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e71d11640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c0ce850 0x7f2e6c141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e71510640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e6c0d0ce0 0x7f2e6c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 -- 172.21.15.106:0/4196395677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c0ce850 msgr2=0x7f2e6c141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c0ce850 0x7f2e6c141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 -- 172.21.15.106:0/4196395677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e6c0d0ce0 msgr2=0x7f2e6c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e6c0d0ce0 0x7f2e6c159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 -- 172.21.15.106:0/4196395677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c1f70f0 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e72512640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c15e0a0 0x7f2e6c15bdb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e640af040 tx=0x7f2e64019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e627fc640 1 -- 172.21.15.106:0/4196395677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640b8670 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e627fc640 1 -- 172.21.15.106:0/4196395677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e640be070 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c145bf0 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e627fc640 1 -- 172.21.15.106:0/4196395677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640b8a30 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.536+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c1461b0 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.540+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6c145960 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.540+0000 7f2e627fc640 1 -- 172.21.15.106:0/4196395677 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2e640b8bd0 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.540+0000 7f2e627fc640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2e5006f430 0x7f2e50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.540+0000 7f2e71d11640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2e5006f430 0x7f2e50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.540+0000 7f2e627fc640 1 -- 172.21.15.106:0/4196395677 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e64129f70 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.540+0000 7f2e71d11640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2e5006f430 0x7f2e50071940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800b930 tx=0x7f2e68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.544+0000 7f2e627fc640 1 -- 172.21.15.106:0/4196395677 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2e640165e0 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.708+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e6c0b3e10 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.708+0000 7f2e627fc640 1 -- 172.21.15.106:0/4196395677 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f2e640f6e70 con 0x7f2e6c15e0a0 2024-04-25T11:19:54.708 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:54.709 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2e5006f430 msgr2=0x7f2e50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2e5006f430 0x7f2e50071940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800b930 tx=0x7f2e68008040 comp rx=0 tx=0).stop 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c15e0a0 msgr2=0x7f2e6c15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c15e0a0 0x7f2e6c15bdb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e640af040 tx=0x7f2e64019790 comp rx=0 tx=0).stop 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 shutdown_connections 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2e5006f430 0x7f2e50071940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e6c15e0a0 0x7f2e6c15bdb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e6c0d0ce0 0x7f2e6c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 --2- 172.21.15.106:0/4196395677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e6c0ce850 0x7f2e6c141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 >> 172.21.15.106:0/4196395677 conn(0x7f2e6c06c7c0 msgr2=0x7f2e6c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:54.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 shutdown_connections 2024-04-25T11:19:54.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.712+0000 7f2e73f75640 1 -- 172.21.15.106:0/4196395677 wait complete. 2024-04-25T11:19:54.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:54.730 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:54.867 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:19:54.870 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:19:54.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cdaff5640 1 Processor -- start 2024-04-25T11:19:54.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cdaff5640 1 -- start start 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cdaff5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cd40d0cf0 0x7f9cd406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cdaff5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9cd415d230 0x7f9cd415d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cdaff5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd40b68e0 con 0x7f9cd40d1740 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cdaff5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd4161210 con 0x7f9cd40d0cf0 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cdaff5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd40b6290 con 0x7f9cd415d230 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd8d91640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cd40d0cf0 0x7f9cd406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd8d91640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cd40d0cf0 0x7f9cd406ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39464/0 (socket says 172.21.15.106:39464) 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd8d91640 1 -- 172.21.15.106:0/2842251607 learned_addr learned my addr 172.21.15.106:0/2842251607 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd9592640 1 --2- 172.21.15.106:0/2842251607 >> v2:172.21.15.134:3300/0 conn(0x7f9cd415d230 0x7f9cd415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd8d91640 1 -- 172.21.15.106:0/2842251607 >> v2:172.21.15.134:3300/0 conn(0x7f9cd415d230 msgr2=0x7f9cd415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:54.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd8d91640 1 --2- 172.21.15.106:0/2842251607 >> v2:172.21.15.134:3300/0 conn(0x7f9cd415d230 0x7f9cd415d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd8d91640 1 -- 172.21.15.106:0/2842251607 >> v1:172.21.15.106:6789/0 conn(0x7f9cd40d1740 legacy=0x7f9cd40d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd37fe640 1 -- 172.21.15.106:0/2842251607 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 39407125 0 0) 0x7f9cd40b68e0 con 0x7f9cd40d1740 2024-04-25T11:19:54.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd8d91640 1 -- 172.21.15.106:0/2842251607 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd40b6570 con 0x7f9cd40d0cf0 2024-04-25T11:19:54.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.872+0000 7f9cd9592640 1 --2- 172.21.15.106:0/2842251607 >> v2:172.21.15.134:3300/0 conn(0x7f9cd415d230 0x7f9cd415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cd8d91640 1 --2- 172.21.15.106:0/2842251607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd40d0cf0 0x7f9cd406ea20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4160400 tx=0x7f9cc401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e6a3925262f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cd37fe640 1 -- 172.21.15.106:0/2842251607 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40a5590 con 0x7f9cd40d0cf0 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cd37fe640 1 -- 172.21.15.106:0/2842251607 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc40a5730 con 0x7f9cd40d0cf0 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cd37fe640 1 -- 172.21.15.106:0/2842251607 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40a5b00 con 0x7f9cd40d0cf0 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cdaff5640 1 -- 172.21.15.106:0/2842251607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd40d0cf0 msgr2=0x7f9cd406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/2842251607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd40d0cf0 0x7f9cd406ea20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4160400 tx=0x7f9cc401c8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cdaff5640 1 -- 172.21.15.106:0/2842251607 shutdown_connections 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/2842251607 >> v2:172.21.15.134:3300/0 conn(0x7f9cd415d230 0x7f9cd415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/2842251607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd40d0cf0 0x7f9cd406ea20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cdaff5640 1 -- 172.21.15.106:0/2842251607 >> 172.21.15.106:0/2842251607 conn(0x7f9cd406c9f0 msgr2=0x7f9cd406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:54.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cdaff5640 1 -- 172.21.15.106:0/2842251607 shutdown_connections 2024-04-25T11:19:54.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.876+0000 7f9cdaff5640 1 -- 172.21.15.106:0/2842251607 wait complete. 2024-04-25T11:19:54.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 Processor -- start 2024-04-25T11:19:54.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 -- start start 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cd40d0cf0 0x7f9cd415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd40d1740 0x7f9cd4149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd415d230 0x7f9cd4072a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd416a080 con 0x7f9cd415d230 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd415cdd0 con 0x7f9cd40d1740 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd416a200 con 0x7f9cd40d0cf0 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd415d230 0x7f9cd4072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd415d230 0x7f9cd4072a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39468/0 (socket says 172.21.15.106:39468) 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 -- 172.21.15.106:0/524339642 learned_addr learned my addr 172.21.15.106:0/524339642 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd3fff640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd40d1740 0x7f9cd4149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd8d91640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cd40d0cf0 0x7f9cd415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 -- 172.21.15.106:0/524339642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cd40d0cf0 msgr2=0x7f9cd415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cd40d0cf0 0x7f9cd415feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 -- 172.21.15.106:0/524339642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd40d1740 msgr2=0x7f9cd4149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd40d1740 0x7f9cd4149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 -- 172.21.15.106:0/524339642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd415df90 con 0x7f9cd415d230 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd9592640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd415d230 0x7f9cd4072a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00c7f0 tx=0x7f9cbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd17fa640 1 -- 172.21.15.106:0/524339642 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc015020 con 0x7f9cd415d230 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd17fa640 1 -- 172.21.15.106:0/524339642 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cbc004250 con 0x7f9cd415d230 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cd17fa640 1 -- 172.21.15.106:0/524339642 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc003620 con 0x7f9cd415d230 2024-04-25T11:19:54.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd415e490 con 0x7f9cd415d230 2024-04-25T11:19:54.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.880+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd415ea50 con 0x7f9cd415d230 2024-04-25T11:19:54.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.884+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd415e1e0 con 0x7f9cd415d230 2024-04-25T11:19:54.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.884+0000 7f9cd17fa640 1 -- 172.21.15.106:0/524339642 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9cbc018020 con 0x7f9cd415d230 2024-04-25T11:19:54.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.884+0000 7f9cd17fa640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9cb406f400 0x7f9cb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.884+0000 7f9cd17fa640 1 -- 172.21.15.106:0/524339642 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9cbc08f250 con 0x7f9cd415d230 2024-04-25T11:19:54.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.884+0000 7f9cd8d91640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9cb406f400 0x7f9cb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.884+0000 7f9cd8d91640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9cb406f400 0x7f9cb4071910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40ae000 tx=0x7f9cc40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:54.888+0000 7f9cd17fa640 1 -- 172.21.15.106:0/524339642 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9cbc01f030 con 0x7f9cd415d230 2024-04-25T11:19:55.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.048+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9cd4050750 con 0x7f9cd415d230 2024-04-25T11:19:55.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.048+0000 7f9cd17fa640 1 -- 172.21.15.106:0/524339642 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f9cbc05c140 con 0x7f9cd415d230 2024-04-25T11:19:55.049 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:19:55.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9cb406f400 msgr2=0x7f9cb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9cb406f400 0x7f9cb4071910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40ae000 tx=0x7f9cc40a7070 comp rx=0 tx=0).stop 2024-04-25T11:19:55.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd415d230 msgr2=0x7f9cd4072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd415d230 0x7f9cd4072a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00c7f0 tx=0x7f9cbc00ccb0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 shutdown_connections 2024-04-25T11:19:55.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9cb406f400 0x7f9cb4071910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cd415d230 0x7f9cd4072a20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd40d1740 0x7f9cd4149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 --2- 172.21.15.106:0/524339642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cd40d0cf0 0x7f9cd415feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 >> 172.21.15.106:0/524339642 conn(0x7f9cd406c9f0 msgr2=0x7f9cd406f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 shutdown_connections 2024-04-25T11:19:55.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.052+0000 7f9cdaff5640 1 -- 172.21.15.106:0/524339642 wait complete. 2024-04-25T11:19:55.070 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:55.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.228+0000 7fc1dfc7b640 1 Processor -- start 2024-04-25T11:19:55.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.228+0000 7fc1dfc7b640 1 -- start start 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dfc7b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80d0cc0 0x7fc1d81644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dfc7b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d8164a10 con 0x7fc1d80b7b00 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dfc7b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d80d1170 con 0x7fc1d80d4c90 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dfc7b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d80d1450 con 0x7fc1d80d0cc0 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1de218640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc1d80b7b00 0x7fc1d80b8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34756/0 (socket says 172.21.15.106:34756) 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1de218640 1 -- 172.21.15.106:0/2719495916 learned_addr learned my addr 172.21.15.106:0/2719495916 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dda17640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80d0cc0 0x7fc1d81644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3840013145 0 0) 0x7fc1d8164a10 con 0x7fc1d80b7b00 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1c4003530 con 0x7fc1d80b7b00 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2498213565 0 0) 0x7fc1c4003530 con 0x7fc1d80b7b00 2024-04-25T11:19:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80d0cc0 msgr2=0x7fc1d81644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80d0cc0 0x7fc1d81644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 >> v1:172.21.15.134:6789/0 conn(0x7fc1d80d4c90 legacy=0x7fc1d80b73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d80d41b0 con 0x7fc1d80b7b00 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fc1d0001a00 con 0x7fc1d80b7b00 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 >> v1:172.21.15.106:6789/0 conn(0x7fc1d80b7b00 legacy=0x7fc1d80b8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c40039f0 0x7fc1c4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80d0cc0 0x7fc1c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c4004d50 0x7fc1c4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d80d1450 con 0x7fc1c4004d50 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c400bb90 con 0x7fc1c40039f0 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c400bd90 con 0x7fc1d80d0cc0 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1de218640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c4004d50 0x7fc1c4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dd216640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80d0cc0 0x7fc1c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dda17640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c40039f0 0x7fc1c4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1de218640 1 -- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80d0cc0 msgr2=0x7fc1c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1de218640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80d0cc0 0x7fc1c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1de218640 1 -- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c40039f0 msgr2=0x7fc1c4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1de218640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c40039f0 0x7fc1c4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1de218640 1 -- 172.21.15.106:0/2719495916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c4003530 con 0x7fc1c4004d50 2024-04-25T11:19:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.232+0000 7fc1dda17640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c40039f0 0x7fc1c4003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1de218640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c4004d50 0x7fc1c4005160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000a830 tx=0x7fc1d001fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44db9506c4bcb49b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d000a480 con 0x7fc1c4004d50 2024-04-25T11:19:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d00aa6f0 con 0x7fc1c4004d50 2024-04-25T11:19:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dca15640 1 -- 172.21.15.106:0/2719495916 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00aab20 con 0x7fc1c4004d50 2024-04-25T11:19:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/2719495916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c4004d50 msgr2=0x7fc1c4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dfc7b640 1 --2- 172.21.15.106:0/2719495916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c4004d50 0x7fc1c4005160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000a830 tx=0x7fc1d001fac0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1de218640 1 -- 172.21.15.106:0/2719495916 reap_dead start 2024-04-25T11:19:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/2719495916 shutdown_connections 2024-04-25T11:19:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/2719495916 >> 172.21.15.106:0/2719495916 conn(0x7fc1d8074480 msgr2=0x7fc1d80d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/2719495916 shutdown_connections 2024-04-25T11:19:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/2719495916 wait complete. 2024-04-25T11:19:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dfc7b640 1 Processor -- start 2024-04-25T11:19:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.236+0000 7fc1dfc7b640 1 -- start start 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80cad50 0x7fc1d80c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8151de0 0x7fc1d80c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80c8cb0 0x7fc1d815d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d80d50a0 con 0x7fc1d8151de0 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d80d09f0 con 0x7fc1d80cad50 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d80d5220 con 0x7fc1d80c8cb0 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8151de0 0x7fc1d80c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8151de0 0x7fc1d80c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39492/0 (socket says 172.21.15.106:39492) 2024-04-25T11:19:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 -- 172.21.15.106:0/3233248115 learned_addr learned my addr 172.21.15.106:0/3233248115 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dda17640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80cad50 0x7fc1d80c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1de218640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80c8cb0 0x7fc1d815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 -- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80c8cb0 msgr2=0x7fc1d815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80c8cb0 0x7fc1d815d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 -- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80cad50 msgr2=0x7fc1d80c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80cad50 0x7fc1d80c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 -- 172.21.15.106:0/3233248115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d80c9cb0 con 0x7fc1d8151de0 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1de218640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80c8cb0 0x7fc1d815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dd216640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8151de0 0x7fc1d80c8770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800e550 tx=0x7fc1c800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1ce7fc640 1 -- 172.21.15.106:0/3233248115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80052e0 con 0x7fc1d8151de0 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1ce7fc640 1 -- 172.21.15.106:0/3233248115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c8018070 con 0x7fc1d8151de0 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d80c9f90 con 0x7fc1d8151de0 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1ce7fc640 1 -- 172.21.15.106:0/3233248115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8003820 con 0x7fc1d8151de0 2024-04-25T11:19:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d80ca550 con 0x7fc1d8151de0 2024-04-25T11:19:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d8050750 con 0x7fc1d8151de0 2024-04-25T11:19:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1ce7fc640 1 -- 172.21.15.106:0/3233248115 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc1c80039c0 con 0x7fc1d8151de0 2024-04-25T11:19:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.240+0000 7fc1ce7fc640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc1bc06f400 0x7fc1bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.244+0000 7fc1ce7fc640 1 -- 172.21.15.106:0/3233248115 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc1c808ef60 con 0x7fc1d8151de0 2024-04-25T11:19:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.244+0000 7fc1dda17640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc1bc06f400 0x7fc1bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.244+0000 7fc1dda17640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc1bc06f400 0x7fc1bc071910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1d815df20 tx=0x7fc1d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.248+0000 7fc1ce7fc640 1 -- 172.21.15.106:0/3233248115 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc1c805bef0 con 0x7fc1d8151de0 2024-04-25T11:19:55.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.408+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1d815eda0 con 0x7fc1d8151de0 2024-04-25T11:19:55.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1ce7fc640 1 -- 172.21.15.106:0/3233248115 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc1c8005480 con 0x7fc1d8151de0 2024-04-25T11:19:55.410 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:19:55.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc1bc06f400 msgr2=0x7fc1bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc1bc06f400 0x7fc1bc071910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1d815df20 tx=0x7fc1d4008040 comp rx=0 tx=0).stop 2024-04-25T11:19:55.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8151de0 msgr2=0x7fc1d80c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8151de0 0x7fc1d80c8770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800e550 tx=0x7fc1c800ea10 comp rx=0 tx=0).stop 2024-04-25T11:19:55.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 shutdown_connections 2024-04-25T11:19:55.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80c8cb0 0x7fc1d815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc1bc06f400 0x7fc1bc071910 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8151de0 0x7fc1d80c8770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 --2- 172.21.15.106:0/3233248115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80cad50 0x7fc1d80c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 >> 172.21.15.106:0/3233248115 conn(0x7fc1d8074480 msgr2=0x7fc1d815e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 shutdown_connections 2024-04-25T11:19:55.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.412+0000 7fc1dfc7b640 1 -- 172.21.15.106:0/3233248115 wait complete. 2024-04-25T11:19:55.431 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:55.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.576+0000 7f46697d9640 1 Processor -- start 2024-04-25T11:19:55.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.576+0000 7f46697d9640 1 -- start start 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46697d9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f46640d1e50 0x7f46640b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46697d9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46640cdd40 con 0x7f4664143fa0 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46697d9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4664073300 con 0x7f46640cd930 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46697d9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f46640735e0 con 0x7f46640d1e50 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f4662ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4664143fa0 0x7f4664072a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41974/0 (socket says 172.21.15.106:41974) 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f4662ffd640 1 -- 172.21.15.106:0/4166774902 learned_addr learned my addr 172.21.15.106:0/4166774902 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46637fe640 1 --2- 172.21.15.106:0/4166774902 >> v2:172.21.15.134:3300/0 conn(0x7f46640d1e50 0x7f46640b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299354317 0 0) 0x7f4654001120 con 0x7f4664143fa0 2024-04-25T11:19:55.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4650003530 con 0x7f4664143fa0 2024-04-25T11:19:55.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2702269305 0 0) 0x7f4650003530 con 0x7f4664143fa0 2024-04-25T11:19:55.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 >> v2:172.21.15.134:3300/0 conn(0x7f46640d1e50 msgr2=0x7f46640b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 --2- 172.21.15.106:0/4166774902 >> v2:172.21.15.134:3300/0 conn(0x7f46640d1e50 0x7f46640b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 >> v1:172.21.15.134:6789/0 conn(0x7f46640cd930 legacy=0x7f46640d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46640738c0 con 0x7f4664143fa0 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f4654001850 con 0x7f4664143fa0 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 >> v1:172.21.15.106:6789/0 conn(0x7f4664143fa0 legacy=0x7f4664072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46500039c0 0x7f4650003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650004810 0x7f4650004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f465000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46640735e0 con 0x7f46500051d0 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f465000c170 con 0x7f4650004810 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f465000c370 con 0x7f46500039c0 2024-04-25T11:19:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46637fe640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f465000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f4662ffd640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46500039c0 0x7f4650003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f465a7fc640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650004810 0x7f4650004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46637fe640 1 -- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46500039c0 msgr2=0x7f4650003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46637fe640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46500039c0 0x7f4650003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46637fe640 1 -- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650004810 msgr2=0x7f4650004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46637fe640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650004810 0x7f4650004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f46637fe640 1 -- 172.21.15.106:0/4166774902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4650003530 con 0x7f46500051d0 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f465a7fc640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650004810 0x7f4650004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.580+0000 7f4662ffd640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46500039c0 0x7f4650003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46637fe640 1 -- 172.21.15.106:0/4166774902 reap_dead start 2024-04-25T11:19:55.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46637fe640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f465000b9d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f465c009f90 tx=0x7f465c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3e8597287296b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c009c10 con 0x7f46500051d0 2024-04-25T11:19:55.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465c0a5bb0 con 0x7f46500051d0 2024-04-25T11:19:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46627fc640 1 -- 172.21.15.106:0/4166774902 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0039a0 con 0x7f46500051d0 2024-04-25T11:19:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 -- 172.21.15.106:0/4166774902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 msgr2=0x7f465000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f465000b9d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f465c009f90 tx=0x7f465c01cbc0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 -- 172.21.15.106:0/4166774902 shutdown_connections 2024-04-25T11:19:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 --2- 172.21.15.106:0/4166774902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f465000b9d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 -- 172.21.15.106:0/4166774902 >> 172.21.15.106:0/4166774902 conn(0x7f466406c9f0 msgr2=0x7f466406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 -- 172.21.15.106:0/4166774902 shutdown_connections 2024-04-25T11:19:55.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 -- 172.21.15.106:0/4166774902 wait complete. 2024-04-25T11:19:55.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 Processor -- start 2024-04-25T11:19:55.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.584+0000 7f46697d9640 1 -- start start 2024-04-25T11:19:55.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f466406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b3a50 0x7f466413fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640b2cd0 0x7f46640b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4664072a20 con 0x7f46500051d0 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46640b5320 con 0x7f46640b3a50 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4664072ba0 con 0x7f46640b2cd0 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f466406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f466406ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45402/0 (socket says 172.21.15.106:45402) 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 -- 172.21.15.106:0/2256255304 learned_addr learned my addr 172.21.15.106:0/2256255304 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 -- 172.21.15.106:0/2256255304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640b2cd0 msgr2=0x7f46640b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640b2cd0 0x7f46640b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 -- 172.21.15.106:0/2256255304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b3a50 msgr2=0x7f466413fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f465a7fc640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b3a50 0x7f466413fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b3a50 0x7f466413fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 -- 172.21.15.106:0/2256255304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46640b3620 con 0x7f46500051d0 2024-04-25T11:19:55.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f465a7fc640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b3a50 0x7f466413fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f4662ffd640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f466406ea20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f465400d770 tx=0x7f465400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f465bfff640 1 -- 172.21.15.106:0/2256255304 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654019070 con 0x7f46500051d0 2024-04-25T11:19:55.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f465bfff640 1 -- 172.21.15.106:0/2256255304 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46540074e0 con 0x7f46500051d0 2024-04-25T11:19:55.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466406f270 con 0x7f46500051d0 2024-04-25T11:19:55.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f465bfff640 1 -- 172.21.15.106:0/2256255304 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46540049f0 con 0x7f46500051d0 2024-04-25T11:19:55.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466406f830 con 0x7f46500051d0 2024-04-25T11:19:55.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4664050750 con 0x7f46500051d0 2024-04-25T11:19:55.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.588+0000 7f465bfff640 1 -- 172.21.15.106:0/2256255304 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f465401c070 con 0x7f46500051d0 2024-04-25T11:19:55.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.592+0000 7f465bfff640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f464406f400 0x7f4644071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.592+0000 7f465a7fc640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f464406f400 0x7f4644071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.592+0000 7f465bfff640 1 -- 172.21.15.106:0/2256255304 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f465408ec60 con 0x7f46500051d0 2024-04-25T11:19:55.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.592+0000 7f465a7fc640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f464406f400 0x7f4644071910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4664142120 tx=0x7f464c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.596+0000 7f465bfff640 1 -- 172.21.15.106:0/2256255304 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f46640b3620 con 0x7f46500051d0 2024-04-25T11:19:55.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.752+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46640b4e20 con 0x7f46500051d0 2024-04-25T11:19:55.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f465bfff640 1 -- 172.21.15.106:0/2256255304 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f46640cdd40 con 0x7f46500051d0 2024-04-25T11:19:55.754 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:19:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f464406f400 msgr2=0x7f4644071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f464406f400 0x7f4644071910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4664142120 tx=0x7f464c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 msgr2=0x7f466406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f466406ea20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f465400d770 tx=0x7f465400dc30 comp rx=0 tx=0).stop 2024-04-25T11:19:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 shutdown_connections 2024-04-25T11:19:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640b2cd0 0x7f46640b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b3a50 0x7f466413fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f464406f400 0x7f4644071910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 --2- 172.21.15.106:0/2256255304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500051d0 0x7f466406ea20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 >> 172.21.15.106:0/2256255304 conn(0x7f466406c9f0 msgr2=0x7f4664142be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 shutdown_connections 2024-04-25T11:19:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.756+0000 7f46697d9640 1 -- 172.21.15.106:0/2256255304 wait complete. 2024-04-25T11:19:55.774 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:55.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.908+0000 7f6232b85640 1 Processor -- start 2024-04-25T11:19:55.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f6232b85640 1 -- start start 2024-04-25T11:19:55.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f6232b85640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f622c15e0a0 0x7f622c073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f6232b85640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f622c0ce850 0x7f622c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f6232b85640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f622c0cf1a0 con 0x7f622c06f5f0 2024-04-25T11:19:55.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f6232b85640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f622c0d1cf0 con 0x7f622c0ce850 2024-04-25T11:19:55.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f6232b85640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f622c0ce050 con 0x7f622c15e0a0 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f622bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f622c0ce850 0x7f622c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f622bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f622c0ce850 0x7f622c0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45406/0 (socket says 172.21.15.106:45406) 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f622bfff640 1 -- 172.21.15.106:0/3031257512 learned_addr learned my addr 172.21.15.106:0/3031257512 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f6230921640 1 --2- 172.21.15.106:0/3031257512 >> v2:172.21.15.134:3300/0 conn(0x7f622c15e0a0 0x7f622c073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f622bfff640 1 -- 172.21.15.106:0/3031257512 >> v2:172.21.15.134:3300/0 conn(0x7f622c15e0a0 msgr2=0x7f622c073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f622bfff640 1 --2- 172.21.15.106:0/3031257512 >> v2:172.21.15.134:3300/0 conn(0x7f622c15e0a0 0x7f622c073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f622bfff640 1 -- 172.21.15.106:0/3031257512 >> v1:172.21.15.134:6789/0 conn(0x7f622c06f5f0 legacy=0x7f622c06fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f622bfff640 1 -- 172.21.15.106:0/3031257512 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c15e6b0 con 0x7f622c0ce850 2024-04-25T11:19:55.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.912+0000 7f6230921640 1 --2- 172.21.15.106:0/3031257512 >> v2:172.21.15.134:3300/0 conn(0x7f622c15e0a0 0x7f622c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f622bfff640 1 --2- 172.21.15.106:0/3031257512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c0ce850 0x7f622c0cec60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f622c0d2420 tx=0x7f621401c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9096ed43870c7fde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f622b7fe640 1 -- 172.21.15.106:0/3031257512 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140a5580 con 0x7f622c0ce850 2024-04-25T11:19:55.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f622b7fe640 1 -- 172.21.15.106:0/3031257512 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62140a5720 con 0x7f622c0ce850 2024-04-25T11:19:55.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f622b7fe640 1 -- 172.21.15.106:0/3031257512 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140a5af0 con 0x7f622c0ce850 2024-04-25T11:19:55.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 -- 172.21.15.106:0/3031257512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c0ce850 msgr2=0x7f622c0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 --2- 172.21.15.106:0/3031257512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c0ce850 0x7f622c0cec60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f622c0d2420 tx=0x7f621401c7d0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 -- 172.21.15.106:0/3031257512 shutdown_connections 2024-04-25T11:19:55.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 --2- 172.21.15.106:0/3031257512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c0ce850 0x7f622c0cec60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 --2- 172.21.15.106:0/3031257512 >> v2:172.21.15.134:3300/0 conn(0x7f622c15e0a0 0x7f622c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 -- 172.21.15.106:0/3031257512 >> 172.21.15.106:0/3031257512 conn(0x7f622c06c7c0 msgr2=0x7f622c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 -- 172.21.15.106:0/3031257512 shutdown_connections 2024-04-25T11:19:55.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 -- 172.21.15.106:0/3031257512 wait complete. 2024-04-25T11:19:55.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 Processor -- start 2024-04-25T11:19:55.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.916+0000 7f6232b85640 1 -- start start 2024-04-25T11:19:55.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c06f5f0 0x7f622c1fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0ce850 0x7f622c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c15e0a0 0x7f622c146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f622c0d0860 con 0x7f622c06f5f0 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f622c0d06e0 con 0x7f622c0ce850 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f622c15f270 con 0x7f622c15e0a0 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c06f5f0 0x7f622c1fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c06f5f0 0x7f622c1fa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45408/0 (socket says 172.21.15.106:45408) 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 -- 172.21.15.106:0/1073020610 learned_addr learned my addr 172.21.15.106:0/1073020610 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f622bfff640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0ce850 0x7f622c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 -- 172.21.15.106:0/1073020610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c15e0a0 msgr2=0x7f622c146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c15e0a0 0x7f622c146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 -- 172.21.15.106:0/1073020610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0ce850 msgr2=0x7f622c15b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0ce850 0x7f622c15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 -- 172.21.15.106:0/1073020610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c1fb490 con 0x7f622c06f5f0 2024-04-25T11:19:55.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6230921640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c06f5f0 0x7f622c1fa9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c820 tx=0x7f621c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f62297fa640 1 -- 172.21.15.106:0/1073020610 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c015070 con 0x7f622c06f5f0 2024-04-25T11:19:55.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f62297fa640 1 -- 172.21.15.106:0/1073020610 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c007a30 con 0x7f622c06f5f0 2024-04-25T11:19:55.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622c1fb770 con 0x7f622c06f5f0 2024-04-25T11:19:55.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f62297fa640 1 -- 172.21.15.106:0/1073020610 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c007e00 con 0x7f622c06f5f0 2024-04-25T11:19:55.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622c1447e0 con 0x7f622c06f5f0 2024-04-25T11:19:55.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622c0b1fd0 con 0x7f622c06f5f0 2024-04-25T11:19:55.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f62297fa640 1 -- 172.21.15.106:0/1073020610 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f621c018070 con 0x7f622c06f5f0 2024-04-25T11:19:55.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f62297fa640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f620c06f430 0x7f620c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f62297fa640 1 -- 172.21.15.106:0/1073020610 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f621c08f250 con 0x7f622c06f5f0 2024-04-25T11:19:55.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f622bfff640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f620c06f430 0x7f620c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.920+0000 7f622bfff640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f620c06f430 0x7f620c071940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f62140ae000 tx=0x7f62140a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:55.928+0000 7f62297fa640 1 -- 172.21.15.106:0/1073020610 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f621c05c140 con 0x7f622c06f5f0 2024-04-25T11:19:56.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.076+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f622c0b1cf0 con 0x7f622c06f5f0 2024-04-25T11:19:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f62297fa640 1 -- 172.21.15.106:0/1073020610 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f621c05ba40 con 0x7f622c06f5f0 2024-04-25T11:19:56.079 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:56.079 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:56.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f620c06f430 msgr2=0x7f620c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f620c06f430 0x7f620c071940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f62140ae000 tx=0x7f62140a7070 comp rx=0 tx=0).stop 2024-04-25T11:19:56.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c06f5f0 msgr2=0x7f622c1fa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c06f5f0 0x7f622c1fa9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c820 tx=0x7f621c00cce0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 shutdown_connections 2024-04-25T11:19:56.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c15e0a0 0x7f622c146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0ce850 0x7f622c15b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f620c06f430 0x7f620c071940 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 --2- 172.21.15.106:0/1073020610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f622c06f5f0 0x7f622c1fa9d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 >> 172.21.15.106:0/1073020610 conn(0x7f622c06c7c0 msgr2=0x7f622c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:56.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 shutdown_connections 2024-04-25T11:19:56.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.080+0000 7f6232b85640 1 -- 172.21.15.106:0/1073020610 wait complete. 2024-04-25T11:19:56.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:56.100 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:56.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.244+0000 7f31cc701640 1 Processor -- start 2024-04-25T11:19:56.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.244+0000 7f31cc701640 1 -- start start 2024-04-25T11:19:56.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31cc701640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f31c40d1e50 0x7f31c406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31cc701640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f31c415d230 0x7f31c415d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31cc701640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c40b68e0 con 0x7f31c40d0cf0 2024-04-25T11:19:56.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31cc701640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f31c4161210 con 0x7f31c40d1e50 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31cc701640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f31c40b6290 con 0x7f31c415d230 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31ca49d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f31c40d0cf0 0x7f31c40d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41976/0 (socket says 172.21.15.106:41976) 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31ca49d640 1 -- 172.21.15.106:0/4211772457 learned_addr learned my addr 172.21.15.106:0/4211772457 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31c9c9c640 1 --2- 172.21.15.106:0/4211772457 >> v2:172.21.15.106:3300/0 conn(0x7f31c40d1e50 0x7f31c406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31cac9e640 1 --2- 172.21.15.106:0/4211772457 >> v2:172.21.15.134:3300/0 conn(0x7f31c415d230 0x7f31c415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31c949b640 1 -- 172.21.15.106:0/4211772457 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804840019 0 0) 0x7f31c40b68e0 con 0x7f31c40d0cf0 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31c949b640 1 -- 172.21.15.106:0/4211772457 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31b0003530 con 0x7f31c40d0cf0 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31c9c9c640 1 -- 172.21.15.106:0/4211772457 >> v2:172.21.15.134:3300/0 conn(0x7f31c415d230 msgr2=0x7f31c415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31c9c9c640 1 --2- 172.21.15.106:0/4211772457 >> v2:172.21.15.134:3300/0 conn(0x7f31c415d230 0x7f31c415d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31c9c9c640 1 -- 172.21.15.106:0/4211772457 >> v1:172.21.15.106:6789/0 conn(0x7f31c40d0cf0 legacy=0x7f31c40d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31c949b640 1 -- 172.21.15.106:0/4211772457 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 386342118 0 0) 0x7f31b0003530 con 0x7f31c40d0cf0 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.248+0000 7f31c9c9c640 1 -- 172.21.15.106:0/4211772457 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c40b6570 con 0x7f31c40d1e50 2024-04-25T11:19:56.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31c9c9c640 1 --2- 172.21.15.106:0/4211772457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c40d1e50 0x7f31c406ea20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f31c4160400 tx=0x7f31b401c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bda5e922b85609b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31c949b640 1 -- 172.21.15.106:0/4211772457 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40b0070 con 0x7f31c40d1e50 2024-04-25T11:19:56.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31c949b640 1 -- 172.21.15.106:0/4211772457 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b40ab400 con 0x7f31c40d1e50 2024-04-25T11:19:56.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31c949b640 1 -- 172.21.15.106:0/4211772457 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0001680 con 0x7f31c40d1e50 2024-04-25T11:19:56.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 -- 172.21.15.106:0/4211772457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c40d1e50 msgr2=0x7f31c406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 --2- 172.21.15.106:0/4211772457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c40d1e50 0x7f31c406ea20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f31c4160400 tx=0x7f31b401c280 comp rx=0 tx=0).stop 2024-04-25T11:19:56.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 -- 172.21.15.106:0/4211772457 shutdown_connections 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 --2- 172.21.15.106:0/4211772457 >> v2:172.21.15.134:3300/0 conn(0x7f31c415d230 0x7f31c415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 --2- 172.21.15.106:0/4211772457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c40d1e50 0x7f31c406ea20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 -- 172.21.15.106:0/4211772457 >> 172.21.15.106:0/4211772457 conn(0x7f31c406c9f0 msgr2=0x7f31c406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 -- 172.21.15.106:0/4211772457 shutdown_connections 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 -- 172.21.15.106:0/4211772457 wait complete. 2024-04-25T11:19:56.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 Processor -- start 2024-04-25T11:19:56.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.252+0000 7f31cc701640 1 -- start start 2024-04-25T11:19:56.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c40d0cf0 0x7f31c414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c415d230 0x7f31c40b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c40b35e0 0x7f31c414fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31c416a080 con 0x7f31c415d230 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31c415cdd0 con 0x7f31c40b35e0 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f31c416a200 con 0x7f31c40d0cf0 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c415d230 0x7f31c40b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c415d230 0x7f31c40b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45436/0 (socket says 172.21.15.106:45436) 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 -- 172.21.15.106:0/2302087839 learned_addr learned my addr 172.21.15.106:0/2302087839 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 -- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c40d0cf0 msgr2=0x7f31c414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31ca49d640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c40d0cf0 0x7f31c414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cac9e640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c40b35e0 0x7f31c414fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c40d0cf0 0x7f31c414ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 -- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c40b35e0 msgr2=0x7f31c414fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c40b35e0 0x7f31c414fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 -- 172.21.15.106:0/2302087839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c41538c0 con 0x7f31c415d230 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cac9e640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c40b35e0 0x7f31c414fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31c9c9c640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c415d230 0x7f31c40b30a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31b40ae040 tx=0x7f31b4019020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31baffd640 1 -- 172.21.15.106:0/2302087839 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40b0040 con 0x7f31c415d230 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31baffd640 1 -- 172.21.15.106:0/2302087839 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b40b9070 con 0x7f31c415d230 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31baffd640 1 -- 172.21.15.106:0/2302087839 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40b4850 con 0x7f31c415d230 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c40b3d90 con 0x7f31c415d230 2024-04-25T11:19:56.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c4156ee0 con 0x7f31c415d230 2024-04-25T11:19:56.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c4050750 con 0x7f31c415d230 2024-04-25T11:19:56.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.256+0000 7f31baffd640 1 -- 172.21.15.106:0/2302087839 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f31b40b49f0 con 0x7f31c415d230 2024-04-25T11:19:56.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.260+0000 7f31baffd640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f31a806f400 0x7f31a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.260+0000 7f31baffd640 1 -- 172.21.15.106:0/2302087839 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f31b4129140 con 0x7f31c415d230 2024-04-25T11:19:56.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.260+0000 7f31ca49d640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f31a806f400 0x7f31a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.260+0000 7f31ca49d640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f31a806f400 0x7f31a8071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31c00017f0 tx=0x7f31c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.264+0000 7f31baffd640 1 -- 172.21.15.106:0/2302087839 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31b4015f90 con 0x7f31c415d230 2024-04-25T11:19:56.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.424+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31c40b3b20 con 0x7f31c415d230 2024-04-25T11:19:56.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.424+0000 7f31baffd640 1 -- 172.21.15.106:0/2302087839 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f31b40f60c0 con 0x7f31c415d230 2024-04-25T11:19:56.426 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:19:56.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.424+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f31a806f400 msgr2=0x7f31a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.424+0000 7f31cc701640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f31a806f400 0x7f31a8071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31c00017f0 tx=0x7f31c0009040 comp rx=0 tx=0).stop 2024-04-25T11:19:56.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.424+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c415d230 msgr2=0x7f31c40b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.424+0000 7f31cc701640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c415d230 0x7f31c40b30a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31b40ae040 tx=0x7f31b4019020 comp rx=0 tx=0).stop 2024-04-25T11:19:56.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.428+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 shutdown_connections 2024-04-25T11:19:56.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.428+0000 7f31cc701640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c40b35e0 0x7f31c414fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.428+0000 7f31cc701640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f31a806f400 0x7f31a8071910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.428+0000 7f31cc701640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c415d230 0x7f31c40b30a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.428+0000 7f31cc701640 1 --2- 172.21.15.106:0/2302087839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c40d0cf0 0x7f31c414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.428+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 >> 172.21.15.106:0/2302087839 conn(0x7f31c406c9f0 msgr2=0x7f31c4159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:56.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.428+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 shutdown_connections 2024-04-25T11:19:56.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.428+0000 7f31cc701640 1 -- 172.21.15.106:0/2302087839 wait complete. 2024-04-25T11:19:56.444 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:56.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff022236640 1 Processor -- start 2024-04-25T11:19:56.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff022236640 1 -- start start 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff022236640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff01c0b2f80 0x7ff01c0b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff022236640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff01c0b8c00 0x7ff01c0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff022236640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff01c0cf770 con 0x7ff01c0b43e0 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff022236640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff01c0cf9e0 con 0x7ff01c0b8c00 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff022236640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff01c06d590 con 0x7ff01c0b2f80 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff01affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff01c0b8c00 0x7ff01c0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff01bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff01c0b43e0 0x7ff01c0d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41992/0 (socket says 172.21.15.106:41992) 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff01bfff640 1 -- 172.21.15.106:0/2637405392 learned_addr learned my addr 172.21.15.106:0/2637405392 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.588+0000 7ff01b7fe640 1 --2- 172.21.15.106:0/2637405392 >> v2:172.21.15.134:3300/0 conn(0x7ff01c0b2f80 0x7ff01c0b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01a7fc640 1 -- 172.21.15.106:0/2637405392 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292380251 0 0) 0x7ff01c0cf770 con 0x7ff01c0b43e0 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01a7fc640 1 -- 172.21.15.106:0/2637405392 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff008003530 con 0x7ff01c0b43e0 2024-04-25T11:19:56.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01affd640 1 -- 172.21.15.106:0/2637405392 >> v2:172.21.15.134:3300/0 conn(0x7ff01c0b2f80 msgr2=0x7ff01c0b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:56.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01affd640 1 --2- 172.21.15.106:0/2637405392 >> v2:172.21.15.134:3300/0 conn(0x7ff01c0b2f80 0x7ff01c0b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01affd640 1 -- 172.21.15.106:0/2637405392 >> v1:172.21.15.106:6789/0 conn(0x7ff01c0b43e0 legacy=0x7ff01c0d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01affd640 1 -- 172.21.15.106:0/2637405392 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c09cd80 con 0x7ff01c0b8c00 2024-04-25T11:19:56.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01b7fe640 1 --2- 172.21.15.106:0/2637405392 >> v2:172.21.15.134:3300/0 conn(0x7ff01c0b2f80 0x7ff01c0b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:56.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01affd640 1 --2- 172.21.15.106:0/2637405392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b8c00 0x7ff01c0b3ea0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff01c051160 tx=0x7ff00401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf74d7176214df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01a7fc640 1 -- 172.21.15.106:0/2637405392 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040b1070 con 0x7ff01c0b8c00 2024-04-25T11:19:56.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01a7fc640 1 -- 172.21.15.106:0/2637405392 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0040a8050 con 0x7ff01c0b8c00 2024-04-25T11:19:56.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff01a7fc640 1 -- 172.21.15.106:0/2637405392 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040ac860 con 0x7ff01c0b8c00 2024-04-25T11:19:56.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff022236640 1 -- 172.21.15.106:0/2637405392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b8c00 msgr2=0x7ff01c0b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff022236640 1 --2- 172.21.15.106:0/2637405392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b8c00 0x7ff01c0b3ea0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff01c051160 tx=0x7ff00401c3c0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff022236640 1 -- 172.21.15.106:0/2637405392 shutdown_connections 2024-04-25T11:19:56.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff022236640 1 --2- 172.21.15.106:0/2637405392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b8c00 0x7ff01c0b3ea0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff022236640 1 --2- 172.21.15.106:0/2637405392 >> v2:172.21.15.134:3300/0 conn(0x7ff01c0b2f80 0x7ff01c0b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff022236640 1 -- 172.21.15.106:0/2637405392 >> 172.21.15.106:0/2637405392 conn(0x7ff01c0d00a0 msgr2=0x7ff01c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:56.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff022236640 1 -- 172.21.15.106:0/2637405392 shutdown_connections 2024-04-25T11:19:56.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.592+0000 7ff022236640 1 -- 172.21.15.106:0/2637405392 wait complete. 2024-04-25T11:19:56.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 Processor -- start 2024-04-25T11:19:56.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 -- start start 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff01c0b2f80 0x7ff01c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b43e0 0x7ff01c1630f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c0b8c00 0x7ff01c06d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c0b4f50 con 0x7ff01c0b43e0 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c0b4dd0 con 0x7ff01c0b8c00 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff01c0b47f0 con 0x7ff01c0b2f80 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b43e0 0x7ff01c1630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b43e0 0x7ff01c1630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45460/0 (socket says 172.21.15.106:45460) 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 -- 172.21.15.106:0/1533865856 learned_addr learned my addr 172.21.15.106:0/1533865856 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01b7fe640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff01c0b2f80 0x7ff01c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 -- 172.21.15.106:0/1533865856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff01c0b2f80 msgr2=0x7ff01c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff01c0b2f80 0x7ff01c0cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 -- 172.21.15.106:0/1533865856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c0b8c00 msgr2=0x7ff01c06d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01bfff640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c0b8c00 0x7ff01c06d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c0b8c00 0x7ff01c06d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 -- 172.21.15.106:0/1533865856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c167fd0 con 0x7ff01c0b43e0 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff01affd640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b43e0 0x7ff01c1630f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff004018fd0 tx=0x7ff00401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff013fff640 1 -- 172.21.15.106:0/1533865856 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040b1040 con 0x7ff01c0b43e0 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff013fff640 1 -- 172.21.15.106:0/1533865856 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0040bc070 con 0x7ff01c0b43e0 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c163a40 con 0x7ff01c0b43e0 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff013fff640 1 -- 172.21.15.106:0/1533865856 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040b7450 con 0x7ff01c0b43e0 2024-04-25T11:19:56.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.596+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1687b0 con 0x7ff01c0b43e0 2024-04-25T11:19:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.600+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01c168240 con 0x7ff01c0b43e0 2024-04-25T11:19:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.600+0000 7ff013fff640 1 -- 172.21.15.106:0/1533865856 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff0040b75f0 con 0x7ff01c0b43e0 2024-04-25T11:19:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.600+0000 7ff013fff640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7feffc06f400 0x7feffc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.600+0000 7ff013fff640 1 -- 172.21.15.106:0/1533865856 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff0041296a0 con 0x7ff01c0b43e0 2024-04-25T11:19:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.600+0000 7ff01b7fe640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7feffc06f400 0x7feffc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.600+0000 7ff01b7fe640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7feffc06f400 0x7feffc071910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009ed0 tx=0x7ff00c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.604+0000 7ff013fff640 1 -- 172.21.15.106:0/1533865856 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff004016070 con 0x7ff01c0b43e0 2024-04-25T11:19:56.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.752+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff01c050750 con 0x7ff01c0b43e0 2024-04-25T11:19:56.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.752+0000 7ff013fff640 1 -- 172.21.15.106:0/1533865856 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff0040f6590 con 0x7ff01c0b43e0 2024-04-25T11:19:56.752 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:19:56.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.752+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7feffc06f400 msgr2=0x7feffc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.752+0000 7ff022236640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7feffc06f400 0x7feffc071910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009ed0 tx=0x7ff00c00b040 comp rx=0 tx=0).stop 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.752+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b43e0 msgr2=0x7ff01c1630f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.752+0000 7ff022236640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b43e0 0x7ff01c1630f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff004018fd0 tx=0x7ff00401ac00 comp rx=0 tx=0).stop 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.756+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 shutdown_connections 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.756+0000 7ff022236640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c0b8c00 0x7ff01c06d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.756+0000 7ff022236640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7feffc06f400 0x7feffc071910 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.756+0000 7ff022236640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff01c0b43e0 0x7ff01c1630f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.756+0000 7ff022236640 1 --2- 172.21.15.106:0/1533865856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff01c0b2f80 0x7ff01c0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.756+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 >> 172.21.15.106:0/1533865856 conn(0x7ff01c0d00a0 msgr2=0x7ff01c070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.756+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 shutdown_connections 2024-04-25T11:19:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.756+0000 7ff022236640 1 -- 172.21.15.106:0/1533865856 wait complete. 2024-04-25T11:19:56.771 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:56.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.924+0000 7f2630d75640 1 Processor -- start 2024-04-25T11:19:56.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.924+0000 7f2630d75640 1 -- start start 2024-04-25T11:19:56.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2630d75640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f262c09c140 0x7f262c0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2630d75640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f262c0cef70 con 0x7f262c0b3170 2024-04-25T11:19:56.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2630d75640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f262c0cf250 con 0x7f262c0b88a0 2024-04-25T11:19:56.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2630d75640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f262c0b2ce0 con 0x7f262c09c140 2024-04-25T11:19:56.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f262ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f262c0b3170 0x7f262c0b3580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42006/0 (socket says 172.21.15.106:42006) 2024-04-25T11:19:56.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f262ad9d640 1 -- 172.21.15.106:0/1999647368 learned_addr learned my addr 172.21.15.106:0/1999647368 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:56.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2629d9b640 1 --2- 172.21.15.106:0/1999647368 >> v2:172.21.15.106:3300/0 conn(0x7f262c09c140 0x7f262c0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2629d9b640 1 -- 172.21.15.106:0/1999647368 >> v1:172.21.15.134:6789/0 conn(0x7f262c0b88a0 legacy=0x7f262c09ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2629d9b640 1 -- 172.21.15.106:0/1999647368 >> v1:172.21.15.106:6789/0 conn(0x7f262c0b3170 legacy=0x7f262c0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2629d9b640 1 -- 172.21.15.106:0/1999647368 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c0cf530 con 0x7f262c09c140 2024-04-25T11:19:56.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f2629d9b640 1 --2- 172.21.15.106:0/1999647368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c09c140 0x7f262c0b2640 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26200097c0 tx=0x7f262001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169cc24c40bda876 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f262959a640 1 -- 172.21.15.106:0/1999647368 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26200b1070 con 0x7f262c09c140 2024-04-25T11:19:56.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f262959a640 1 -- 172.21.15.106:0/1999647368 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26200ac410 con 0x7f262c09c140 2024-04-25T11:19:56.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.928+0000 7f262959a640 1 -- 172.21.15.106:0/1999647368 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c0b44a0 con 0x7f262c09c140 2024-04-25T11:19:56.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- 172.21.15.106:0/1999647368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c09c140 msgr2=0x7f262c0b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 --2- 172.21.15.106:0/1999647368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c09c140 0x7f262c0b2640 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26200097c0 tx=0x7f262001c4a0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- 172.21.15.106:0/1999647368 shutdown_connections 2024-04-25T11:19:56.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 --2- 172.21.15.106:0/1999647368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c09c140 0x7f262c0b2640 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- 172.21.15.106:0/1999647368 >> 172.21.15.106:0/1999647368 conn(0x7f262c0b5060 msgr2=0x7f262c0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:56.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- 172.21.15.106:0/1999647368 shutdown_connections 2024-04-25T11:19:56.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- 172.21.15.106:0/1999647368 wait complete. 2024-04-25T11:19:56.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 Processor -- start 2024-04-25T11:19:56.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- start start 2024-04-25T11:19:56.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f262c0b3170 0x7f262c152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c0b88a0 0x7f262c0c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f262c071460 0x7f262c071870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f262c16c5b0 con 0x7f262c0b88a0 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f262c09cf60 con 0x7f262c0b3170 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2630d75640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f262c16c730 con 0x7f262c071460 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2629d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c0b88a0 0x7f262c0c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2629d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c0b88a0 0x7f262c0c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45478/0 (socket says 172.21.15.106:45478) 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.932+0000 7f2629d9b640 1 -- 172.21.15.106:0/993064373 learned_addr learned my addr 172.21.15.106:0/993064373 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f262a59c640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f262c0b3170 0x7f262c152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f262ad9d640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f262c071460 0x7f262c071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2629d9b640 1 -- 172.21.15.106:0/993064373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f262c071460 msgr2=0x7f262c071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2629d9b640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f262c071460 0x7f262c071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2629d9b640 1 -- 172.21.15.106:0/993064373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f262c0b3170 msgr2=0x7f262c152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2629d9b640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f262c0b3170 0x7f262c152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2629d9b640 1 -- 172.21.15.106:0/993064373 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c071db0 con 0x7f262c0b88a0 2024-04-25T11:19:56.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2629d9b640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c0b88a0 0x7f262c0c7560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26200190b0 tx=0x7f2620019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f261affd640 1 -- 172.21.15.106:0/993064373 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26200b1040 con 0x7f262c0b88a0 2024-04-25T11:19:56.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f261affd640 1 -- 172.21.15.106:0/993064373 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26200b5410 con 0x7f262c0b88a0 2024-04-25T11:19:56.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c0c7aa0 con 0x7f262c0b88a0 2024-04-25T11:19:56.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f261affd640 1 -- 172.21.15.106:0/993064373 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26200b55e0 con 0x7f262c0b88a0 2024-04-25T11:19:56.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1538e0 con 0x7f262c0b88a0 2024-04-25T11:19:56.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262c050750 con 0x7f262c0b88a0 2024-04-25T11:19:56.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f261affd640 1 -- 172.21.15.106:0/993064373 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f26200bd020 con 0x7f262c0b88a0 2024-04-25T11:19:56.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f261affd640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f260c06f400 0x7f260c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:56.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f261affd640 1 -- 172.21.15.106:0/993064373 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f26201286a0 con 0x7f262c0b88a0 2024-04-25T11:19:56.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f262a59c640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f260c06f400 0x7f260c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:56.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.936+0000 7f262a59c640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f260c06f400 0x7f260c071910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f261c001600 tx=0x7f261c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:56.940+0000 7f261affd640 1 -- 172.21.15.106:0/993064373 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26200c4020 con 0x7f262c0b88a0 2024-04-25T11:19:57.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.096+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f262c159a70 con 0x7f262c0b88a0 2024-04-25T11:19:57.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.096+0000 7f261affd640 1 -- 172.21.15.106:0/993064373 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f262c071db0 con 0x7f262c0b88a0 2024-04-25T11:19:57.096 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f260c06f400 msgr2=0x7f260c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f260c06f400 0x7f260c071910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f261c001600 tx=0x7f261c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c0b88a0 msgr2=0x7f262c0c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c0b88a0 0x7f262c0c7560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26200190b0 tx=0x7f2620019240 comp rx=0 tx=0).stop 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 shutdown_connections 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f262c071460 0x7f262c071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f260c06f400 0x7f260c071910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f262c0b88a0 0x7f262c0c7560 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 --2- 172.21.15.106:0/993064373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f262c0b3170 0x7f262c152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 >> 172.21.15.106:0/993064373 conn(0x7f262c0b5060 msgr2=0x7f262c0d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:57.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 shutdown_connections 2024-04-25T11:19:57.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.100+0000 7f2630d75640 1 -- 172.21.15.106:0/993064373 wait complete. 2024-04-25T11:19:57.116 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:57.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d910e640 1 Processor -- start 2024-04-25T11:19:57.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d910e640 1 -- start start 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d910e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09d40ba970 0x7f09d40d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d910e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f09d40d2160 0x7f09d40d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d910e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d40d2ab0 con 0x7f09d40b5120 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d910e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f09d40d1100 con 0x7f09d40ba970 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d910e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f09d40d0530 con 0x7f09d40d2160 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09d40ba970 0x7f09d40d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d359e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f09d40b5120 0x7f09d40b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42008/0 (socket says 172.21.15.106:42008) 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d359e640 1 -- 172.21.15.106:0/2379712500 learned_addr learned my addr 172.21.15.106:0/2379712500 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d259c640 1 --2- 172.21.15.106:0/2379712500 >> v2:172.21.15.134:3300/0 conn(0x7f09d40d2160 0x7f09d40d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d2d9d640 1 -- 172.21.15.106:0/2379712500 >> v2:172.21.15.134:3300/0 conn(0x7f09d40d2160 msgr2=0x7f09d40d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d2d9d640 1 --2- 172.21.15.106:0/2379712500 >> v2:172.21.15.134:3300/0 conn(0x7f09d40d2160 0x7f09d40d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d2d9d640 1 -- 172.21.15.106:0/2379712500 >> v1:172.21.15.106:6789/0 conn(0x7f09d40b5120 legacy=0x7f09d40b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.260+0000 7f09d2d9d640 1 -- 172.21.15.106:0/2379712500 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d40b5c40 con 0x7f09d40ba970 2024-04-25T11:19:57.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d2d9d640 1 --2- 172.21.15.106:0/2379712500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d40d1590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09d40d68c0 tx=0x7f09c401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17834d30fcaa558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d1d9b640 1 -- 172.21.15.106:0/2379712500 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c40b0070 con 0x7f09d40ba970 2024-04-25T11:19:57.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d1d9b640 1 -- 172.21.15.106:0/2379712500 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c40ab600 con 0x7f09d40ba970 2024-04-25T11:19:57.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d1d9b640 1 -- 172.21.15.106:0/2379712500 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c40ab9d0 con 0x7f09d40ba970 2024-04-25T11:19:57.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 -- 172.21.15.106:0/2379712500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 msgr2=0x7f09d40d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 --2- 172.21.15.106:0/2379712500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d40d1590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09d40d68c0 tx=0x7f09c401c460 comp rx=0 tx=0).stop 2024-04-25T11:19:57.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 -- 172.21.15.106:0/2379712500 shutdown_connections 2024-04-25T11:19:57.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 --2- 172.21.15.106:0/2379712500 >> v2:172.21.15.134:3300/0 conn(0x7f09d40d2160 0x7f09d40d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 --2- 172.21.15.106:0/2379712500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d40d1590 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 -- 172.21.15.106:0/2379712500 >> 172.21.15.106:0/2379712500 conn(0x7f09d40d57e0 msgr2=0x7f09d40d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:57.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 -- 172.21.15.106:0/2379712500 shutdown_connections 2024-04-25T11:19:57.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 -- 172.21.15.106:0/2379712500 wait complete. 2024-04-25T11:19:57.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.264+0000 7f09d910e640 1 Processor -- start 2024-04-25T11:19:57.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 -- start start 2024-04-25T11:19:57.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09d40b5120 0x7f09d41573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d4163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d40d2160 0x7f09d40b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09d40b4da0 con 0x7f09d40ba970 2024-04-25T11:19:57.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09d40d1ad0 con 0x7f09d40d2160 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09d40b4f20 con 0x7f09d40b5120 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d4163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d4163280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45496/0 (socket says 172.21.15.106:45496) 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 -- 172.21.15.106:0/2109665312 learned_addr learned my addr 172.21.15.106:0/2109665312 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 -- 172.21.15.106:0/2109665312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09d40b5120 msgr2=0x7f09d41573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09d40b5120 0x7f09d41573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 -- 172.21.15.106:0/2109665312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d40d2160 msgr2=0x7f09d40b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d40d2160 0x7f09d40b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 -- 172.21.15.106:0/2109665312 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d41578f0 con 0x7f09d40ba970 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d259c640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d4163280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f09c800d7d0 tx=0x7f09c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09bb7fe640 1 -- 172.21.15.106:0/2109665312 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c800b6c0 con 0x7f09d40ba970 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09bb7fe640 1 -- 172.21.15.106:0/2109665312 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c8019070 con 0x7f09d40ba970 2024-04-25T11:19:57.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d415b610 con 0x7f09d40ba970 2024-04-25T11:19:57.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09bb7fe640 1 -- 172.21.15.106:0/2109665312 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c800baa0 con 0x7f09d40ba970 2024-04-25T11:19:57.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d415bbd0 con 0x7f09d40ba970 2024-04-25T11:19:57.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d4160290 con 0x7f09d40ba970 2024-04-25T11:19:57.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.268+0000 7f09bb7fe640 1 -- 172.21.15.106:0/2109665312 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f09c800bc40 con 0x7f09d40ba970 2024-04-25T11:19:57.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.272+0000 7f09bb7fe640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09c006f400 0x7f09c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.272+0000 7f09bb7fe640 1 -- 172.21.15.106:0/2109665312 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f09c808f770 con 0x7f09d40ba970 2024-04-25T11:19:57.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.272+0000 7f09d2d9d640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09c006f400 0x7f09c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.272+0000 7f09d2d9d640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09c006f400 0x7f09c0071910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09c40ad000 tx=0x7f09c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.276+0000 7f09bb7fe640 1 -- 172.21.15.106:0/2109665312 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09c805c660 con 0x7f09d40ba970 2024-04-25T11:19:57.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.424+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09d415b360 con 0x7f09d40ba970 2024-04-25T11:19:57.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.428+0000 7f09bb7fe640 1 -- 172.21.15.106:0/2109665312 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f09c805bf60 con 0x7f09d40ba970 2024-04-25T11:19:57.429 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:57.429 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09c006f400 msgr2=0x7f09c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09c006f400 0x7f09c0071910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09c40ad000 tx=0x7f09c4008040 comp rx=0 tx=0).stop 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 msgr2=0x7f09d4163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d4163280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f09c800d7d0 tx=0x7f09c800dc90 comp rx=0 tx=0).stop 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 shutdown_connections 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d40d2160 0x7f09d40b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09c006f400 0x7f09c0071910 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09d40ba970 0x7f09d4163280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 --2- 172.21.15.106:0/2109665312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09d40b5120 0x7f09d41573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 >> 172.21.15.106:0/2109665312 conn(0x7f09d40d57e0 msgr2=0x7f09d40d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:57.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 shutdown_connections 2024-04-25T11:19:57.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.432+0000 7f09d910e640 1 -- 172.21.15.106:0/2109665312 wait complete. 2024-04-25T11:19:57.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:57.449 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:57.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f334dc63640 1 Processor -- start 2024-04-25T11:19:57.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f334dc63640 1 -- start start 2024-04-25T11:19:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f334dc63640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3348143fa0 0x7f33480cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f334dc63640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3348072a20 0x7f33480d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f334dc63640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3348073300 con 0x7f3348072e30 2024-04-25T11:19:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f334dc63640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f33480735e0 con 0x7f3348072a20 2024-04-25T11:19:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f334dc63640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f33480738c0 con 0x7f3348143fa0 2024-04-25T11:19:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f3346ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3348072a20 0x7f33480d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f3347fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3348072e30 0x7f33480b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42014/0 (socket says 172.21.15.106:42014) 2024-04-25T11:19:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f3347fff640 1 -- 172.21.15.106:0/1153661600 learned_addr learned my addr 172.21.15.106:0/1153661600 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f33477fe640 1 --2- 172.21.15.106:0/1153661600 >> v2:172.21.15.134:3300/0 conn(0x7f3348143fa0 0x7f33480cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f3346ffd640 1 -- 172.21.15.106:0/1153661600 >> v2:172.21.15.134:3300/0 conn(0x7f3348143fa0 msgr2=0x7f33480cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f3346ffd640 1 --2- 172.21.15.106:0/1153661600 >> v2:172.21.15.134:3300/0 conn(0x7f3348143fa0 0x7f33480cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f3346ffd640 1 -- 172.21.15.106:0/1153661600 >> v1:172.21.15.106:6789/0 conn(0x7f3348072e30 legacy=0x7f33480b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.588+0000 7f3346ffd640 1 -- 172.21.15.106:0/1153661600 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33480b5040 con 0x7f3348072a20 2024-04-25T11:19:57.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f3346ffd640 1 --2- 172.21.15.106:0/1153661600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348072a20 0x7f33480d1740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3348070f30 tx=0x7f333c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f42f40e2c40148a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f33467fc640 1 -- 172.21.15.106:0/1153661600 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0b0070 con 0x7f3348072a20 2024-04-25T11:19:57.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f33467fc640 1 -- 172.21.15.106:0/1153661600 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c0ab600 con 0x7f3348072a20 2024-04-25T11:19:57.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f33467fc640 1 -- 172.21.15.106:0/1153661600 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0ab9d0 con 0x7f3348072a20 2024-04-25T11:19:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 -- 172.21.15.106:0/1153661600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348072a20 msgr2=0x7f33480d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 --2- 172.21.15.106:0/1153661600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348072a20 0x7f33480d1740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3348070f30 tx=0x7f333c01c310 comp rx=0 tx=0).stop 2024-04-25T11:19:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 -- 172.21.15.106:0/1153661600 shutdown_connections 2024-04-25T11:19:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 --2- 172.21.15.106:0/1153661600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348072a20 0x7f33480d1740 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 --2- 172.21.15.106:0/1153661600 >> v2:172.21.15.134:3300/0 conn(0x7f3348143fa0 0x7f33480cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 -- 172.21.15.106:0/1153661600 >> 172.21.15.106:0/1153661600 conn(0x7f334806c9f0 msgr2=0x7f334806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 -- 172.21.15.106:0/1153661600 shutdown_connections 2024-04-25T11:19:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 -- 172.21.15.106:0/1153661600 wait complete. 2024-04-25T11:19:57.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 Processor -- start 2024-04-25T11:19:57.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.592+0000 7f334dc63640 1 -- start start 2024-04-25T11:19:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3348072a20 0x7f334815c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3348072e30 0x7f33481e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348143fa0 0x7f33481e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f33480d1e00 con 0x7f3348143fa0 2024-04-25T11:19:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33480d1c80 con 0x7f3348072e30 2024-04-25T11:19:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f33480d1f80 con 0x7f3348072a20 2024-04-25T11:19:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348143fa0 0x7f33481e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348143fa0 0x7f33481e81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45514/0 (socket says 172.21.15.106:45514) 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 -- 172.21.15.106:0/845170187 learned_addr learned my addr 172.21.15.106:0/845170187 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3346ffd640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3348072e30 0x7f33481e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f33477fe640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3348072a20 0x7f334815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 -- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3348072a20 msgr2=0x7f334815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3348072a20 0x7f334815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 -- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3348072e30 msgr2=0x7f33481e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3348072e30 0x7f33481e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 -- 172.21.15.106:0/845170187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33481e6bf0 con 0x7f3348143fa0 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f33477fe640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3348072a20 0x7f334815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f3347fff640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348143fa0 0x7f33481e81d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f334000e550 tx=0x7f334000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f332ffff640 1 -- 172.21.15.106:0/845170187 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340009500 con 0x7f3348143fa0 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f332ffff640 1 -- 172.21.15.106:0/845170187 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3340018070 con 0x7f3348143fa0 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33481e6ed0 con 0x7f3348143fa0 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f332ffff640 1 -- 172.21.15.106:0/845170187 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33400098e0 con 0x7f3348143fa0 2024-04-25T11:19:57.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33481e7490 con 0x7f3348143fa0 2024-04-25T11:19:57.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33481e6960 con 0x7f3348143fa0 2024-04-25T11:19:57.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.596+0000 7f332ffff640 1 -- 172.21.15.106:0/845170187 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3340013480 con 0x7f3348143fa0 2024-04-25T11:19:57.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.600+0000 7f332ffff640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f333406f400 0x7f3334071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.600+0000 7f33477fe640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f333406f400 0x7f3334071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.600+0000 7f332ffff640 1 -- 172.21.15.106:0/845170187 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3340092450 con 0x7f3348143fa0 2024-04-25T11:19:57.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.600+0000 7f33477fe640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f333406f400 0x7f3334071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33380097a0 tx=0x7f3338008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.604+0000 7f332ffff640 1 -- 172.21.15.106:0/845170187 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f334005f3d0 con 0x7f3348143fa0 2024-04-25T11:19:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f33480b4e20 con 0x7f3348143fa0 2024-04-25T11:19:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f332ffff640 1 -- 172.21.15.106:0/845170187 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f334005ecd0 con 0x7f3348143fa0 2024-04-25T11:19:57.751 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f333406f400 msgr2=0x7f3334071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f333406f400 0x7f3334071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33380097a0 tx=0x7f3338008040 comp rx=0 tx=0).stop 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348143fa0 msgr2=0x7f33481e81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348143fa0 0x7f33481e81d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f334000e550 tx=0x7f334000ea10 comp rx=0 tx=0).stop 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 shutdown_connections 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f333406f400 0x7f3334071910 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3348143fa0 0x7f33481e81d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3348072e30 0x7f33481e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 --2- 172.21.15.106:0/845170187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3348072a20 0x7f334815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.752+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 >> 172.21.15.106:0/845170187 conn(0x7f334806c9f0 msgr2=0x7f33480cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.756+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 shutdown_connections 2024-04-25T11:19:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.756+0000 7f334dc63640 1 -- 172.21.15.106:0/845170187 wait complete. 2024-04-25T11:19:57.771 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:57.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.908+0000 7f2730a36640 1 Processor -- start 2024-04-25T11:19:57.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.908+0000 7f2730a36640 1 -- start start 2024-04-25T11:19:57.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f2730a36640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f272c0d0cf0 0x7f272c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f2730a36640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 0x7f272c0d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f2730a36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f272c0d2090 con 0x7f272c15d230 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f2730a36640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f272c0b68e0 con 0x7f272c0d0cf0 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f2730a36640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f272c161210 con 0x7f272c0d1740 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f272c15d230 0x7f272c15d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42028/0 (socket says 172.21.15.106:42028) 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272ad9d640 1 -- 172.21.15.106:0/660156229 learned_addr learned my addr 172.21.15.106:0/660156229 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2969112565 0 0) 0x7f27240014a0 con 0x7f272c15d230 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2714003530 con 0x7f272c15d230 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272a59c640 1 --2- 172.21.15.106:0/660156229 >> v2:172.21.15.134:3300/0 conn(0x7f272c0d0cf0 0x7f272c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f2729d9b640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 0x7f272c0d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 21506605 0 0) 0x7f2714003530 con 0x7f272c15d230 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 msgr2=0x7f272c0d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:57.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 0x7f272c0d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 >> v2:172.21.15.134:3300/0 conn(0x7f272c0d0cf0 msgr2=0x7f272c06ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 --2- 172.21.15.106:0/660156229 >> v2:172.21.15.134:3300/0 conn(0x7f272c0d0cf0 0x7f272c06ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c0b6290 con 0x7f272c15d230 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f2724001bd0 con 0x7f272c15d230 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 >> v1:172.21.15.106:6789/0 conn(0x7f272c15d230 legacy=0x7f272c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f2714003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 0x7f2714004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2714004d50 0x7f2714005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f272c0b68e0 con 0x7f27140039c0 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f272c161210 con 0x7f2714004d50 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f271400bbb0 con 0x7f272c0d1740 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272a59c640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f2714003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f2729d9b640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 0x7f2714004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272a59c640 1 -- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 msgr2=0x7f2714004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:57.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272a59c640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 0x7f2714004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272a59c640 1 -- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2714004d50 msgr2=0x7f2714005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:57.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272a59c640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2714004d50 0x7f2714005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f272a59c640 1 -- 172.21.15.106:0/660156229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2714003530 con 0x7f27140039c0 2024-04-25T11:19:57.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.912+0000 7f2729d9b640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0d1740 0x7f2714004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:57.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f272a59c640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f2714003e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f271c009ab0 tx=0x7f271c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4393db76347ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0096b0 con 0x7f27140039c0 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c0a6430 con 0x7f27140039c0 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f272959a640 1 -- 172.21.15.106:0/660156229 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0a67e0 con 0x7f27140039c0 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f2730a36640 1 -- 172.21.15.106:0/660156229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 msgr2=0x7f2714003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f2730a36640 1 --2- 172.21.15.106:0/660156229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f2714003e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f271c009ab0 tx=0x7f271c01ca50 comp rx=0 tx=0).stop 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f272ad9d640 1 -- 172.21.15.106:0/660156229 reap_dead start 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f2730a36640 1 -- 172.21.15.106:0/660156229 shutdown_connections 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f2730a36640 1 -- 172.21.15.106:0/660156229 >> 172.21.15.106:0/660156229 conn(0x7f272c06c9f0 msgr2=0x7f272c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f2730a36640 1 -- 172.21.15.106:0/660156229 shutdown_connections 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f2730a36640 1 -- 172.21.15.106:0/660156229 wait complete. 2024-04-25T11:19:57.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f2730a36640 1 Processor -- start 2024-04-25T11:19:57.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.916+0000 7f2730a36640 1 -- start start 2024-04-25T11:19:57.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f272c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c152720 0x7f272c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cfa50 0x7f272c154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f272c15cd40 con 0x7f27140039c0 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f272c15cbc0 con 0x7f272c152720 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f272c167e60 con 0x7f272c0cfa50 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f272c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f272c152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45532/0 (socket says 172.21.15.106:45532) 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 -- 172.21.15.106:0/2865981273 learned_addr learned my addr 172.21.15.106:0/2865981273 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272ad9d640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cfa50 0x7f272c154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 -- 172.21.15.106:0/2865981273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cfa50 msgr2=0x7f272c154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cfa50 0x7f272c154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 -- 172.21.15.106:0/2865981273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c152720 msgr2=0x7f272c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c152720 0x7f272c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 -- 172.21.15.106:0/2865981273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c0cfe60 con 0x7f27140039c0 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f272a59c640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f272c152160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f271c01c880 tx=0x7f2714005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f271affd640 1 -- 172.21.15.106:0/2865981273 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0a6c60 con 0x7f27140039c0 2024-04-25T11:19:57.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f271affd640 1 -- 172.21.15.106:0/2865981273 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c0a6e00 con 0x7f27140039c0 2024-04-25T11:19:57.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272c15fa00 con 0x7f27140039c0 2024-04-25T11:19:57.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f271affd640 1 -- 172.21.15.106:0/2865981273 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0ba680 con 0x7f27140039c0 2024-04-25T11:19:57.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272c1622f0 con 0x7f27140039c0 2024-04-25T11:19:57.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f272c050750 con 0x7f27140039c0 2024-04-25T11:19:57.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f271affd640 1 -- 172.21.15.106:0/2865981273 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f271c0ba820 con 0x7f27140039c0 2024-04-25T11:19:57.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f271affd640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f270c06f430 0x7f270c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f2729d9b640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f270c06f430 0x7f270c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.920+0000 7f271affd640 1 -- 172.21.15.106:0/2865981273 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f271c129180 con 0x7f27140039c0 2024-04-25T11:19:57.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.924+0000 7f2729d9b640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f270c06f430 0x7f270c071940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2720006aa0 tx=0x7f2720006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:57.928+0000 7f271affd640 1 -- 172.21.15.106:0/2865981273 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f272c0cfe60 con 0x7f27140039c0 2024-04-25T11:19:58.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.072+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f272c06fae0 con 0x7f27140039c0 2024-04-25T11:19:58.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.072+0000 7f271affd640 1 -- 172.21.15.106:0/2865981273 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f271c0ac020 con 0x7f27140039c0 2024-04-25T11:19:58.073 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:19:58.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f270c06f430 msgr2=0x7f270c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f270c06f430 0x7f270c071940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2720006aa0 tx=0x7f2720006a30 comp rx=0 tx=0).stop 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 msgr2=0x7f272c152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f272c152160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f271c01c880 tx=0x7f2714005840 comp rx=0 tx=0).stop 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 shutdown_connections 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cfa50 0x7f272c154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c152720 0x7f272c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f270c06f430 0x7f270c071940 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 --2- 172.21.15.106:0/2865981273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27140039c0 0x7f272c152160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 >> 172.21.15.106:0/2865981273 conn(0x7f272c06c9f0 msgr2=0x7f272c166f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 shutdown_connections 2024-04-25T11:19:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.076+0000 7f2730a36640 1 -- 172.21.15.106:0/2865981273 wait complete. 2024-04-25T11:19:58.094 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b300c640 1 Processor -- start 2024-04-25T11:19:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b300c640 1 -- start start 2024-04-25T11:19:58.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b300c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac0d1e50 0x7fe8ac1644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b300c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe8ac0d0cb0 0x7fe8ac166a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b300c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8ac06ec70 con 0x7fe8ac0cf4d0 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b300c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe8ac0707b0 con 0x7fe8ac0d0cb0 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b300c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ac06f880 con 0x7fe8ac0d1e50 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b0da8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe8ac0cf4d0 0x7fe8ac0d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42040/0 (socket says 172.21.15.106:42040) 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b0da8640 1 -- 172.21.15.106:0/2892009076 learned_addr learned my addr 172.21.15.106:0/2892009076 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b15a9640 1 --2- 172.21.15.106:0/2892009076 >> v2:172.21.15.134:3300/0 conn(0x7fe8ac0d0cb0 0x7fe8ac166a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8abfff640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac0d1e50 0x7fe8ac1644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788374935 0 0) 0x7fe8ac06ec70 con 0x7fe8ac0cf4d0 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe89c003530 con 0x7fe8ac0cf4d0 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1589272060 0 0) 0x7fe89c003530 con 0x7fe8ac0cf4d0 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac0d1e50 msgr2=0x7fe8ac1644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac0d1e50 0x7fe8ac1644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 >> v2:172.21.15.134:3300/0 conn(0x7fe8ac0d0cb0 msgr2=0x7fe8ac166a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 --2- 172.21.15.106:0/2892009076 >> v2:172.21.15.134:3300/0 conn(0x7fe8ac0d0cb0 0x7fe8ac166a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac06ef50 con 0x7fe8ac0cf4d0 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fe8a0001680 con 0x7fe8ac0cf4d0 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 >> v1:172.21.15.106:6789/0 conn(0x7fe8ac0cf4d0 legacy=0x7fe8ac0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe89c0039c0 0x7fe89c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe89c004350 0x7fe89c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe89c004ee0 0x7fe89c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00bf10 con 0x7fe89c004ee0 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00c090 con 0x7fe89c004350 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c00c210 con 0x7fe89c0039c0 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b15a9640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe89c004ee0 0x7fe89c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b0da8640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe89c0039c0 0x7fe89c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8abfff640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe89c004350 0x7fe89c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b15a9640 1 -- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe89c0039c0 msgr2=0x7fe89c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:58.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b15a9640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe89c0039c0 0x7fe89c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b15a9640 1 -- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe89c004350 msgr2=0x7fe89c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:58.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b15a9640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe89c004350 0x7fe89c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8b15a9640 1 -- 172.21.15.106:0/2892009076 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c003530 con 0x7fe89c004ee0 2024-04-25T11:19:58.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.236+0000 7fe8abfff640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe89c004350 0x7fe89c004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:58.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8b15a9640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe89c004ee0 0x7fe89c00b9d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0d2400 tx=0x7fe8a401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95547e2967269db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40b2070 con 0x7fe89c004ee0 2024-04-25T11:19:58.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a40a5b60 con 0x7fe89c004ee0 2024-04-25T11:19:58.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8ab7fe640 1 -- 172.21.15.106:0/2892009076 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4009c10 con 0x7fe89c004ee0 2024-04-25T11:19:58.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8b300c640 1 -- 172.21.15.106:0/2892009076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe89c004ee0 msgr2=0x7fe89c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8b300c640 1 --2- 172.21.15.106:0/2892009076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe89c004ee0 0x7fe89c00b9d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0d2400 tx=0x7fe8a401cc60 comp rx=0 tx=0).stop 2024-04-25T11:19:58.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8b15a9640 1 -- 172.21.15.106:0/2892009076 reap_dead start 2024-04-25T11:19:58.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8b300c640 1 -- 172.21.15.106:0/2892009076 shutdown_connections 2024-04-25T11:19:58.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8b300c640 1 -- 172.21.15.106:0/2892009076 >> 172.21.15.106:0/2892009076 conn(0x7fe8ac06c9f0 msgr2=0x7fe8ac06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:58.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8b300c640 1 -- 172.21.15.106:0/2892009076 shutdown_connections 2024-04-25T11:19:58.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.240+0000 7fe8b300c640 1 -- 172.21.15.106:0/2892009076 wait complete. 2024-04-25T11:19:58.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 Processor -- start 2024-04-25T11:19:58.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 -- start start 2024-04-25T11:19:58.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8ac14f310 0x7fe8ac1e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac14eb80 0x7fe8ac0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8ac072a20 0x7fe8ac072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac06f510 con 0x7fe8ac072a20 2024-04-25T11:19:58.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac06ea20 con 0x7fe8ac14f310 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ac0d1e50 con 0x7fe8ac14eb80 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8ac072a20 0x7fe8ac072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8ac072a20 0x7fe8ac072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45542/0 (socket says 172.21.15.106:45542) 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 -- 172.21.15.106:0/3376392291 learned_addr learned my addr 172.21.15.106:0/3376392291 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b0da8640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8ac14f310 0x7fe8ac1e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8abfff640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac14eb80 0x7fe8ac0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 -- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac14eb80 msgr2=0x7fe8ac0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac14eb80 0x7fe8ac0af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 -- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8ac14f310 msgr2=0x7fe8ac1e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8ac14f310 0x7fe8ac1e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 -- 172.21.15.106:0/3376392291 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac0b2e80 con 0x7fe8ac072a20 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8abfff640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac14eb80 0x7fe8ac0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:58.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b15a9640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8ac072a20 0x7fe8ac072e30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8a401d500 tx=0x7fe8a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8a97fa640 1 -- 172.21.15.106:0/3376392291 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40b2040 con 0x7fe8ac072a20 2024-04-25T11:19:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8a97fa640 1 -- 172.21.15.106:0/3376392291 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a40bc070 con 0x7fe8ac072a20 2024-04-25T11:19:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac073370 con 0x7fe8ac072a20 2024-04-25T11:19:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8a97fa640 1 -- 172.21.15.106:0/3376392291 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4009c10 con 0x7fe8ac072a20 2024-04-25T11:19:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac0b0120 con 0x7fe8ac072a20 2024-04-25T11:19:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ac050750 con 0x7fe8ac072a20 2024-04-25T11:19:58.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.244+0000 7fe8a97fa640 1 -- 172.21.15.106:0/3376392291 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe8a40a5d10 con 0x7fe8ac072a20 2024-04-25T11:19:58.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.248+0000 7fe8a97fa640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe89006f400 0x7fe890071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.248+0000 7fe8a97fa640 1 -- 172.21.15.106:0/3376392291 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8a41298f0 con 0x7fe8ac072a20 2024-04-25T11:19:58.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.248+0000 7fe8b0da8640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe89006f400 0x7fe890071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.248+0000 7fe8b0da8640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe89006f400 0x7fe890071910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0000fc0 tx=0x7fe8a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.252+0000 7fe8a97fa640 1 -- 172.21.15.106:0/3376392291 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe8ac073370 con 0x7fe8ac072a20 2024-04-25T11:19:58.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8ac158630 con 0x7fe8ac072a20 2024-04-25T11:19:58.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8a97fa640 1 -- 172.21.15.106:0/3376392291 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe8a4016970 con 0x7fe8ac072a20 2024-04-25T11:19:58.407 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:19:58.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe89006f400 msgr2=0x7fe890071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe89006f400 0x7fe890071910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0000fc0 tx=0x7fe8a0009040 comp rx=0 tx=0).stop 2024-04-25T11:19:58.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8ac072a20 msgr2=0x7fe8ac072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8ac072a20 0x7fe8ac072e30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8a401d500 tx=0x7fe8a4009750 comp rx=0 tx=0).stop 2024-04-25T11:19:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 shutdown_connections 2024-04-25T11:19:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe89006f400 0x7fe890071910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8ac072a20 0x7fe8ac072e30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8ac14eb80 0x7fe8ac0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 --2- 172.21.15.106:0/3376392291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8ac14f310 0x7fe8ac1e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 >> 172.21.15.106:0/3376392291 conn(0x7fe8ac06c9f0 msgr2=0x7fe8ac0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 shutdown_connections 2024-04-25T11:19:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.408+0000 7fe8b300c640 1 -- 172.21.15.106:0/3376392291 wait complete. 2024-04-25T11:19:58.427 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:58.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f8124b01640 1 Processor -- start 2024-04-25T11:19:58.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f8124b01640 1 -- start start 2024-04-25T11:19:58.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f8124b01640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f81200d1d00 0x7f81200d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f8124b01640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1190 0x7f81200d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f8124b01640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f81200b6410 0x7f81200b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f8124b01640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f81200d0fc0 con 0x7f81200d1d00 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f8124b01640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f81200b2700 con 0x7f81200b6410 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f8124b01640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f81200d0ce0 con 0x7f81200d1190 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f81200d1d00 0x7f81200d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f81200d1d00 0x7f81200d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45552/0 (socket says 172.21.15.106:45552) 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811e59c640 1 -- 172.21.15.106:0/765393685 learned_addr learned my addr 172.21.15.106:0/765393685 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811ed9d640 1 --2- 172.21.15.106:0/765393685 >> v2:172.21.15.134:3300/0 conn(0x7f81200b6410 0x7f81200b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811dd9b640 1 --2- 172.21.15.106:0/765393685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1190 0x7f81200d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811e59c640 1 -- 172.21.15.106:0/765393685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1190 msgr2=0x7f81200d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811e59c640 1 --2- 172.21.15.106:0/765393685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1190 0x7f81200d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811e59c640 1 -- 172.21.15.106:0/765393685 >> v2:172.21.15.134:3300/0 conn(0x7f81200b6410 msgr2=0x7f81200b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811e59c640 1 --2- 172.21.15.106:0/765393685 >> v2:172.21.15.134:3300/0 conn(0x7f81200b6410 0x7f81200b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.568+0000 7f811e59c640 1 -- 172.21.15.106:0/765393685 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81200b8d00 con 0x7f81200d1d00 2024-04-25T11:19:58.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f811e59c640 1 --2- 172.21.15.106:0/765393685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200d1d00 0x7f81200d2110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8120074a50 tx=0x7f811801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b9680c85d9e7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f811d59a640 1 -- 172.21.15.106:0/765393685 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180ab5f0 con 0x7f81200d1d00 2024-04-25T11:19:58.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f811d59a640 1 -- 172.21.15.106:0/765393685 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81180b0070 con 0x7f81200d1d00 2024-04-25T11:19:58.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f811d59a640 1 -- 172.21.15.106:0/765393685 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180ab9b0 con 0x7f81200d1d00 2024-04-25T11:19:58.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 -- 172.21.15.106:0/765393685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200d1d00 msgr2=0x7f81200d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 --2- 172.21.15.106:0/765393685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200d1d00 0x7f81200d2110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8120074a50 tx=0x7f811801c520 comp rx=0 tx=0).stop 2024-04-25T11:19:58.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 -- 172.21.15.106:0/765393685 shutdown_connections 2024-04-25T11:19:58.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 --2- 172.21.15.106:0/765393685 >> v2:172.21.15.134:3300/0 conn(0x7f81200b6410 0x7f81200b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 --2- 172.21.15.106:0/765393685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1190 0x7f81200d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 --2- 172.21.15.106:0/765393685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200d1d00 0x7f81200d2110 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 -- 172.21.15.106:0/765393685 >> 172.21.15.106:0/765393685 conn(0x7f81200d0040 msgr2=0x7f81200d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:58.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 -- 172.21.15.106:0/765393685 shutdown_connections 2024-04-25T11:19:58.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.572+0000 7f8124b01640 1 -- 172.21.15.106:0/765393685 wait complete. 2024-04-25T11:19:58.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 Processor -- start 2024-04-25T11:19:58.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 -- start start 2024-04-25T11:19:58.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200b6410 0x7f8120152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81200d1190 0x7f8120160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1d00 0x7f8120160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8120162970 con 0x7f81200b6410 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f81201627f0 con 0x7f81200d1190 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8120162af0 con 0x7f81200d1d00 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200b6410 0x7f8120152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200b6410 0x7f8120152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45556/0 (socket says 172.21.15.106:45556) 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 -- 172.21.15.106:0/2491125860 learned_addr learned my addr 172.21.15.106:0/2491125860 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811dd9b640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81200d1190 0x7f8120160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811ed9d640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1d00 0x7f8120160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 -- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1d00 msgr2=0x7f8120160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1d00 0x7f8120160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 -- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81200d1190 msgr2=0x7f8120160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81200d1190 0x7f8120160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 -- 172.21.15.106:0/2491125860 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812006f250 con 0x7f81200b6410 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811dd9b640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81200d1190 0x7f8120160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f811e59c640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200b6410 0x7f8120152e90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f811801abd0 tx=0x7f811801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8116ffd640 1 -- 172.21.15.106:0/2491125860 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180be5f0 con 0x7f81200b6410 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8116ffd640 1 -- 172.21.15.106:0/2491125860 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81180be790 con 0x7f81200b6410 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8120168360 con 0x7f81200b6410 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8116ffd640 1 -- 172.21.15.106:0/2491125860 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180beb80 con 0x7f81200b6410 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.576+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8120168620 con 0x7f81200b6410 2024-04-25T11:19:58.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.580+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812006efd0 con 0x7f81200b6410 2024-04-25T11:19:58.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.580+0000 7f8116ffd640 1 -- 172.21.15.106:0/2491125860 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f81180b43f0 con 0x7f81200b6410 2024-04-25T11:19:58.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.580+0000 7f8116ffd640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f810006f400 0x7f8100071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.580+0000 7f8116ffd640 1 -- 172.21.15.106:0/2491125860 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8118129c50 con 0x7f81200b6410 2024-04-25T11:19:58.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.580+0000 7f811dd9b640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f810006f400 0x7f8100071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.580+0000 7f811dd9b640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f810006f400 0x7f8100071910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f81200d1610 tx=0x7f810800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.584+0000 7f8116ffd640 1 -- 172.21.15.106:0/2491125860 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f812006efd0 con 0x7f81200b6410 2024-04-25T11:19:58.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.744+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81200b83e0 con 0x7f81200b6410 2024-04-25T11:19:58.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.744+0000 7f8116ffd640 1 -- 172.21.15.106:0/2491125860 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f811812f020 con 0x7f81200b6410 2024-04-25T11:19:58.745 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:19:58.745 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f810006f400 msgr2=0x7f8100071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f810006f400 0x7f8100071910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f81200d1610 tx=0x7f810800b040 comp rx=0 tx=0).stop 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200b6410 msgr2=0x7f8120152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200b6410 0x7f8120152e90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f811801abd0 tx=0x7f811801ad60 comp rx=0 tx=0).stop 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 shutdown_connections 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81200d1d00 0x7f8120160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81200d1190 0x7f8120160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f810006f400 0x7f8100071910 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 --2- 172.21.15.106:0/2491125860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81200b6410 0x7f8120152e90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 >> 172.21.15.106:0/2491125860 conn(0x7f81200d0040 msgr2=0x7f81200b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:58.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 shutdown_connections 2024-04-25T11:19:58.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.748+0000 7f8124b01640 1 -- 172.21.15.106:0/2491125860 wait complete. 2024-04-25T11:19:58.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:58.766 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff86e54a640 1 Processor -- start 2024-04-25T11:19:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff86e54a640 1 -- start start 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff86e54a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8680d1230 0x7ff8680d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff86e54a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff8680d2150 0x7ff8680b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff86e54a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8680b91b0 con 0x7ff8680b3060 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff86e54a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff8680b25b0 con 0x7ff8680d1230 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff86e54a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff8680d0c00 con 0x7ff8680d2150 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff867fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8680d1230 0x7ff8680d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff867fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8680d1230 0x7ff8680d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45562/0 (socket says 172.21.15.106:45562) 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff867fff640 1 -- 172.21.15.106:0/1681860279 learned_addr learned my addr 172.21.15.106:0/1681860279 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.912+0000 7ff8677fe640 1 --2- 172.21.15.106:0/1681860279 >> v2:172.21.15.134:3300/0 conn(0x7ff8680d2150 0x7ff8680b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff867fff640 1 -- 172.21.15.106:0/1681860279 >> v2:172.21.15.134:3300/0 conn(0x7ff8680d2150 msgr2=0x7ff8680b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff867fff640 1 --2- 172.21.15.106:0/1681860279 >> v2:172.21.15.134:3300/0 conn(0x7ff8680d2150 0x7ff8680b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff867fff640 1 -- 172.21.15.106:0/1681860279 >> v1:172.21.15.134:6789/0 conn(0x7ff8680b3060 legacy=0x7ff868161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:58.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff867fff640 1 -- 172.21.15.106:0/1681860279 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86809cd80 con 0x7ff8680d1230 2024-04-25T11:19:58.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff867fff640 1 --2- 172.21.15.106:0/1681860279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d1230 0x7ff8680d1c10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff868074200 tx=0x7ff85801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405399325f902708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff866ffd640 1 -- 172.21.15.106:0/1681860279 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580b0070 con 0x7ff8680d1230 2024-04-25T11:19:58.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff866ffd640 1 -- 172.21.15.106:0/1681860279 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8580ab600 con 0x7ff8680d1230 2024-04-25T11:19:58.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff866ffd640 1 -- 172.21.15.106:0/1681860279 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580ab9d0 con 0x7ff8680d1230 2024-04-25T11:19:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff86e54a640 1 -- 172.21.15.106:0/1681860279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d1230 msgr2=0x7ff8680d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff86e54a640 1 --2- 172.21.15.106:0/1681860279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d1230 0x7ff8680d1c10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff868074200 tx=0x7ff85801c410 comp rx=0 tx=0).stop 2024-04-25T11:19:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff86e54a640 1 -- 172.21.15.106:0/1681860279 shutdown_connections 2024-04-25T11:19:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff86e54a640 1 --2- 172.21.15.106:0/1681860279 >> v2:172.21.15.134:3300/0 conn(0x7ff8680d2150 0x7ff8680b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff86e54a640 1 --2- 172.21.15.106:0/1681860279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d1230 0x7ff8680d1c10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff86e54a640 1 -- 172.21.15.106:0/1681860279 >> 172.21.15.106:0/1681860279 conn(0x7ff8680d00e0 msgr2=0x7ff8680d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff86e54a640 1 -- 172.21.15.106:0/1681860279 shutdown_connections 2024-04-25T11:19:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.916+0000 7ff86e54a640 1 -- 172.21.15.106:0/1681860279 wait complete. 2024-04-25T11:19:58.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 Processor -- start 2024-04-25T11:19:58.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 -- start start 2024-04-25T11:19:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8680b3060 0x7ff86806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8680d1230 0x7ff868155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d2150 0x7ff868158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680cfed0 con 0x7ff8680d2150 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680cfd50 con 0x7ff8680b3060 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680d1640 con 0x7ff8680d1230 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d2150 0x7ff868158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d2150 0x7ff868158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45568/0 (socket says 172.21.15.106:45568) 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 -- 172.21.15.106:0/3922850273 learned_addr learned my addr 172.21.15.106:0/3922850273 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff867fff640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8680b3060 0x7ff86806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff8677fe640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8680d1230 0x7ff868155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 -- 172.21.15.106:0/3922850273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8680d1230 msgr2=0x7ff868155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8680d1230 0x7ff868155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 -- 172.21.15.106:0/3922850273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8680b3060 msgr2=0x7ff86806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8680b3060 0x7ff86806e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 -- 172.21.15.106:0/3922850273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86806d830 con 0x7ff8680d2150 2024-04-25T11:19:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86cae7640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d2150 0x7ff868158580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b970 tx=0x7ff86000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff864ff9640 1 -- 172.21.15.106:0/3922850273 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860019070 con 0x7ff8680d2150 2024-04-25T11:19:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff864ff9640 1 -- 172.21.15.106:0/3922850273 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86000cae0 con 0x7ff8680d2150 2024-04-25T11:19:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86806db10 con 0x7ff8680d2150 2024-04-25T11:19:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff864ff9640 1 -- 172.21.15.106:0/3922850273 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860014440 con 0x7ff8680d2150 2024-04-25T11:19:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86806e0d0 con 0x7ff8680d2150 2024-04-25T11:19:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.920+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff868050750 con 0x7ff8680d2150 2024-04-25T11:19:58.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.924+0000 7ff864ff9640 1 -- 172.21.15.106:0/3922850273 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff86001c070 con 0x7ff8680d2150 2024-04-25T11:19:58.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.924+0000 7ff864ff9640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff84806f400 0x7ff848071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:58.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.924+0000 7ff864ff9640 1 -- 172.21.15.106:0/3922850273 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff86008ef20 con 0x7ff8680d2150 2024-04-25T11:19:58.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.924+0000 7ff867fff640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff84806f400 0x7ff848071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:58.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.924+0000 7ff867fff640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff84806f400 0x7ff848071910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff8580ad000 tx=0x7ff8580a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:58.928+0000 7ff864ff9640 1 -- 172.21.15.106:0/3922850273 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff86005bea0 con 0x7ff8680d2150 2024-04-25T11:19:59.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8680cf780 con 0x7ff8680d2150 2024-04-25T11:19:59.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff864ff9640 1 -- 172.21.15.106:0/3922850273 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ff86005b7a0 con 0x7ff8680d2150 2024-04-25T11:19:59.084 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:19:59.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff84806f400 msgr2=0x7ff848071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff84806f400 0x7ff848071910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff8580ad000 tx=0x7ff8580a6000 comp rx=0 tx=0).stop 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d2150 msgr2=0x7ff868158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d2150 0x7ff868158580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b970 tx=0x7ff86000be30 comp rx=0 tx=0).stop 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 shutdown_connections 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff84806f400 0x7ff848071910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8680d2150 0x7ff868158580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8680d1230 0x7ff868155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 --2- 172.21.15.106:0/3922850273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8680b3060 0x7ff86806e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 >> 172.21.15.106:0/3922850273 conn(0x7ff8680d00e0 msgr2=0x7ff868070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 shutdown_connections 2024-04-25T11:19:59.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.084+0000 7ff86e54a640 1 -- 172.21.15.106:0/3922850273 wait complete. 2024-04-25T11:19:59.103 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:59.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85c772640 1 Processor -- start 2024-04-25T11:19:59.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85c772640 1 -- start start 2024-04-25T11:19:59.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85c772640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd854143fa0 0x7fd8540cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85c772640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 0x7fd8540d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85c772640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd854073300 con 0x7fd854072e30 2024-04-25T11:19:59.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85c772640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd8540735e0 con 0x7fd854143fa0 2024-04-25T11:19:59.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85c772640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8540738c0 con 0x7fd854072a20 2024-04-25T11:19:59.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85a50e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd854143fa0 0x7fd8540cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85ad0f640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd854072e30 0x7fd8540b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42050/0 (socket says 172.21.15.106:42050) 2024-04-25T11:19:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85ad0f640 1 -- 172.21.15.106:0/3994015027 learned_addr learned my addr 172.21.15.106:0/3994015027 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd859d0d640 1 --2- 172.21.15.106:0/3994015027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 0x7fd8540d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85a50e640 1 -- 172.21.15.106:0/3994015027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 msgr2=0x7fd8540d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85a50e640 1 --2- 172.21.15.106:0/3994015027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 0x7fd8540d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85a50e640 1 -- 172.21.15.106:0/3994015027 >> v1:172.21.15.106:6789/0 conn(0x7fd854072e30 legacy=0x7fd8540b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85950c640 1 -- 172.21.15.106:0/3994015027 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 957235915 0 0) 0x7fd854073300 con 0x7fd854072e30 2024-04-25T11:19:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.244+0000 7fd85a50e640 1 -- 172.21.15.106:0/3994015027 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8540b5040 con 0x7fd854143fa0 2024-04-25T11:19:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85a50e640 1 --2- 172.21.15.106:0/3994015027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854143fa0 0x7fd8540cd930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd8500097c0 tx=0x7fd85001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd43f4fd7c065fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85950c640 1 -- 172.21.15.106:0/3994015027 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8500b2070 con 0x7fd854143fa0 2024-04-25T11:19:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85950c640 1 -- 172.21.15.106:0/3994015027 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8500a9070 con 0x7fd854143fa0 2024-04-25T11:19:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85950c640 1 -- 172.21.15.106:0/3994015027 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8500ad880 con 0x7fd854143fa0 2024-04-25T11:19:59.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 -- 172.21.15.106:0/3994015027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854143fa0 msgr2=0x7fd8540cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 --2- 172.21.15.106:0/3994015027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854143fa0 0x7fd8540cd930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd8500097c0 tx=0x7fd85001c470 comp rx=0 tx=0).stop 2024-04-25T11:19:59.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 -- 172.21.15.106:0/3994015027 shutdown_connections 2024-04-25T11:19:59.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 --2- 172.21.15.106:0/3994015027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 0x7fd8540d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 --2- 172.21.15.106:0/3994015027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854143fa0 0x7fd8540cd930 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 -- 172.21.15.106:0/3994015027 >> 172.21.15.106:0/3994015027 conn(0x7fd85406c9f0 msgr2=0x7fd85406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:59.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 -- 172.21.15.106:0/3994015027 shutdown_connections 2024-04-25T11:19:59.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 -- 172.21.15.106:0/3994015027 wait complete. 2024-04-25T11:19:59.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 Processor -- start 2024-04-25T11:19:59.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.248+0000 7fd85c772640 1 -- start start 2024-04-25T11:19:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 0x7fd8541559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854072e30 0x7fd8540b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd854143fa0 0x7fd8540b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd85416ad20 con 0x7fd854072e30 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd85416aba0 con 0x7fd854143fa0 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd85416aea0 con 0x7fd854072a20 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854072e30 0x7fd8540b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854072e30 0x7fd8540b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45590/0 (socket says 172.21.15.106:45590) 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 -- 172.21.15.106:0/656524517 learned_addr learned my addr 172.21.15.106:0/656524517 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85ad0f640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd854143fa0 0x7fd8540b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85a50e640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 0x7fd8541559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 -- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 msgr2=0x7fd8541559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 0x7fd8541559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 -- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd854143fa0 msgr2=0x7fd8540b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd854143fa0 0x7fd8540b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 -- 172.21.15.106:0/656524517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd854155f30 con 0x7fd854072e30 2024-04-25T11:19:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85ad0f640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd854143fa0 0x7fd8540b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:59.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd859d0d640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854072e30 0x7fd8540b39b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd84400d5f0 tx=0x7fd84400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd84affd640 1 -- 172.21.15.106:0/656524517 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440125f0 con 0x7fd854072e30 2024-04-25T11:19:59.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd84affd640 1 -- 172.21.15.106:0/656524517 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844019070 con 0x7fd854072e30 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd84affd640 1 -- 172.21.15.106:0/656524517 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440129b0 con 0x7fd854072e30 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8540b4450 con 0x7fd854072e30 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85415f500 con 0x7fd854072e30 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8540b41c0 con 0x7fd854072e30 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd84affd640 1 -- 172.21.15.106:0/656524517 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd844012b50 con 0x7fd854072e30 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd84affd640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd83806f400 0x7fd838071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd84affd640 1 -- 172.21.15.106:0/656524517 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd84408f660 con 0x7fd854072e30 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.252+0000 7fd85a50e640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd83806f400 0x7fd838071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.256+0000 7fd85a50e640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd83806f400 0x7fd838071910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8500af000 tx=0x7fd8500a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.260+0000 7fd84affd640 1 -- 172.21.15.106:0/656524517 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd84405c5e0 con 0x7fd854072e30 2024-04-25T11:19:59.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd854050750 con 0x7fd854072e30 2024-04-25T11:19:59.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd84affd640 1 -- 172.21.15.106:0/656524517 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd84405bee0 con 0x7fd854072e30 2024-04-25T11:19:59.407 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:19:59.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd83806f400 msgr2=0x7fd838071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd83806f400 0x7fd838071910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8500af000 tx=0x7fd8500a7000 comp rx=0 tx=0).stop 2024-04-25T11:19:59.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854072e30 msgr2=0x7fd8540b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854072e30 0x7fd8540b39b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd84400d5f0 tx=0x7fd84400dab0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 shutdown_connections 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd854143fa0 0x7fd8540b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd83806f400 0x7fd838071910 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd854072e30 0x7fd8540b39b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 --2- 172.21.15.106:0/656524517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd854072a20 0x7fd8541559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 >> 172.21.15.106:0/656524517 conn(0x7fd85406c9f0 msgr2=0x7fd854143500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 shutdown_connections 2024-04-25T11:19:59.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.408+0000 7fd85c772640 1 -- 172.21.15.106:0/656524517 wait complete. 2024-04-25T11:19:59.427 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:59.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc72bcab640 1 Processor -- start 2024-04-25T11:19:59.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc72bcab640 1 -- start start 2024-04-25T11:19:59.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc72bcab640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7240b6120 0x7fc72415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc72bcab640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc724073130 0x7fc72415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc72bcab640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72415c520 con 0x7fc7240706c0 2024-04-25T11:19:59.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc72bcab640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc7240b6530 con 0x7fc7240b6120 2024-04-25T11:19:59.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc72bcab640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc7240b5000 con 0x7fc724073130 2024-04-25T11:19:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc729a47640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7240b6120 0x7fc72415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc729a47640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7240b6120 0x7fc72415bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45594/0 (socket says 172.21.15.106:45594) 2024-04-25T11:19:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc729a47640 1 -- 172.21.15.106:0/280123528 learned_addr learned my addr 172.21.15.106:0/280123528 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc72a248640 1 --2- 172.21.15.106:0/280123528 >> v2:172.21.15.134:3300/0 conn(0x7fc724073130 0x7fc72415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc729a47640 1 -- 172.21.15.106:0/280123528 >> v2:172.21.15.134:3300/0 conn(0x7fc724073130 msgr2=0x7fc72415fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc729a47640 1 --2- 172.21.15.106:0/280123528 >> v2:172.21.15.134:3300/0 conn(0x7fc724073130 0x7fc72415fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc729a47640 1 -- 172.21.15.106:0/280123528 >> v1:172.21.15.134:6789/0 conn(0x7fc7240706c0 legacy=0x7fc724072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.576+0000 7fc729a47640 1 -- 172.21.15.106:0/280123528 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7240b43f0 con 0x7fc7240b6120 2024-04-25T11:19:59.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc729a47640 1 --2- 172.21.15.106:0/280123528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240b6120 0x7fc72415bfe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc7240724b0 tx=0x7fc72001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73820bc0efd69a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc728a45640 1 -- 172.21.15.106:0/280123528 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7200ab430 con 0x7fc7240b6120 2024-04-25T11:19:59.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc728a45640 1 -- 172.21.15.106:0/280123528 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7200b0070 con 0x7fc7240b6120 2024-04-25T11:19:59.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc728a45640 1 -- 172.21.15.106:0/280123528 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7200ab7f0 con 0x7fc7240b6120 2024-04-25T11:19:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc72bcab640 1 -- 172.21.15.106:0/280123528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240b6120 msgr2=0x7fc72415bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc72bcab640 1 --2- 172.21.15.106:0/280123528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240b6120 0x7fc72415bfe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc7240724b0 tx=0x7fc72001c4b0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc72bcab640 1 -- 172.21.15.106:0/280123528 shutdown_connections 2024-04-25T11:19:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc72bcab640 1 --2- 172.21.15.106:0/280123528 >> v2:172.21.15.134:3300/0 conn(0x7fc724073130 0x7fc72415fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc72bcab640 1 --2- 172.21.15.106:0/280123528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240b6120 0x7fc72415bfe0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc72bcab640 1 -- 172.21.15.106:0/280123528 >> 172.21.15.106:0/280123528 conn(0x7fc72406c9f0 msgr2=0x7fc72406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc72bcab640 1 -- 172.21.15.106:0/280123528 shutdown_connections 2024-04-25T11:19:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.580+0000 7fc72bcab640 1 -- 172.21.15.106:0/280123528 wait complete. 2024-04-25T11:19:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 Processor -- start 2024-04-25T11:19:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 -- start start 2024-04-25T11:19:59.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240706c0 0x7fc7241499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7240b6120 0x7fc7241462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7241f5ea0 0x7fc72414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc724070c50 con 0x7fc7240706c0 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc724070ad0 con 0x7fc7241f5ea0 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc7241601b0 con 0x7fc7240b6120 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240706c0 0x7fc7241499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240706c0 0x7fc7241499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45596/0 (socket says 172.21.15.106:45596) 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 -- 172.21.15.106:0/1477402421 learned_addr learned my addr 172.21.15.106:0/1477402421 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 -- 172.21.15.106:0/1477402421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7240b6120 msgr2=0x7fc7241462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7240b6120 0x7fc7241462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 -- 172.21.15.106:0/1477402421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7241f5ea0 msgr2=0x7fc72414e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7241f5ea0 0x7fc72414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 -- 172.21.15.106:0/1477402421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7240b2240 con 0x7fc7240706c0 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc729a47640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240706c0 0x7fc7241499c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7200ad040 tx=0x7fc720019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc7167fc640 1 -- 172.21.15.106:0/1477402421 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7200b4520 con 0x7fc7240706c0 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc7167fc640 1 -- 172.21.15.106:0/1477402421 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7200b46e0 con 0x7fc7240706c0 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7240b2520 con 0x7fc7240706c0 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc7167fc640 1 -- 172.21.15.106:0/1477402421 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7200b4ad0 con 0x7fc7240706c0 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7240b2ae0 con 0x7fc7240706c0 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.584+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc724050750 con 0x7fc7240706c0 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.588+0000 7fc7167fc640 1 -- 172.21.15.106:0/1477402421 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc7200c3060 con 0x7fc7240706c0 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.588+0000 7fc7167fc640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc70806f430 0x7fc708071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.588+0000 7fc729246640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc70806f430 0x7fc708071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.588+0000 7fc7167fc640 1 -- 172.21.15.106:0/1477402421 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc72012aaf0 con 0x7fc7240706c0 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.588+0000 7fc729246640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc70806f430 0x7fc708071940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7240b1c00 tx=0x7fc718006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.592+0000 7fc7167fc640 1 -- 172.21.15.106:0/1477402421 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7200160d0 con 0x7fc7240706c0 2024-04-25T11:19:59.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.748+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc72406fec0 con 0x7fc7240706c0 2024-04-25T11:19:59.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.748+0000 7fc7167fc640 1 -- 172.21.15.106:0/1477402421 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc7200f7a70 con 0x7fc7240706c0 2024-04-25T11:19:59.747 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:19:59.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc70806f430 msgr2=0x7fc708071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc70806f430 0x7fc708071940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7240b1c00 tx=0x7fc718006bd0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240706c0 msgr2=0x7fc7241499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240706c0 0x7fc7241499c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7200ad040 tx=0x7fc720019250 comp rx=0 tx=0).stop 2024-04-25T11:19:59.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 shutdown_connections 2024-04-25T11:19:59.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7241f5ea0 0x7fc72414e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7240b6120 0x7fc7241462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc70806f430 0x7fc708071940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 --2- 172.21.15.106:0/1477402421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7240706c0 0x7fc7241499c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 >> 172.21.15.106:0/1477402421 conn(0x7fc72406c9f0 msgr2=0x7fc72414fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:59.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 shutdown_connections 2024-04-25T11:19:59.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.752+0000 7fc72bcab640 1 -- 172.21.15.106:0/1477402421 wait complete. 2024-04-25T11:19:59.788 INFO:tasks.cephfs_test_runner:Starting test: test_flush (tasks.cephfs.test_readahead.TestReadahead) 2024-04-25T11:19:59.788 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:59.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f214397d640 1 Processor -- start 2024-04-25T11:19:59.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f214397d640 1 -- start start 2024-04-25T11:19:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f214397d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c06cd00 0x7f213c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f214397d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f213c0cf990 0x7f213c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f214397d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c0af370 con 0x7f213c15dd50 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f214397d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f213c15e160 con 0x7f213c0cf990 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f214397d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f213c0d2110 con 0x7f213c06cd00 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2141f1a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f213c0cf990 0x7f213c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2141719640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f213c15dd50 0x7f213c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42062/0 (socket says 172.21.15.106:42062) 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2141719640 1 -- 172.21.15.106:0/2318697012 learned_addr learned my addr 172.21.15.106:0/2318697012 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2140f18640 1 --2- 172.21.15.106:0/2318697012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c06cd00 0x7f213c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2141f1a640 1 -- 172.21.15.106:0/2318697012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c06cd00 msgr2=0x7f213c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2141f1a640 1 --2- 172.21.15.106:0/2318697012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c06cd00 0x7f213c0d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2141f1a640 1 -- 172.21.15.106:0/2318697012 >> v1:172.21.15.106:6789/0 conn(0x7f213c15dd50 legacy=0x7f213c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2141f1a640 1 -- 172.21.15.106:0/2318697012 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c0af650 con 0x7f213c0cf990 2024-04-25T11:19:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.932+0000 7f2140f18640 1 --2- 172.21.15.106:0/2318697012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c06cd00 0x7f213c0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f2141f1a640 1 --2- 172.21.15.106:0/2318697012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c0cf990 0x7f213c0cfda0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2134009dc0 tx=0x7f213401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0aef8bb3be40e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f2133fff640 1 -- 172.21.15.106:0/2318697012 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340b2070 con 0x7f213c0cf990 2024-04-25T11:19:59.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f2133fff640 1 -- 172.21.15.106:0/2318697012 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21340a5950 con 0x7f213c0cf990 2024-04-25T11:19:59.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f2133fff640 1 -- 172.21.15.106:0/2318697012 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340a5d20 con 0x7f213c0cf990 2024-04-25T11:19:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f214397d640 1 -- 172.21.15.106:0/2318697012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c0cf990 msgr2=0x7f213c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f214397d640 1 --2- 172.21.15.106:0/2318697012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c0cf990 0x7f213c0cfda0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2134009dc0 tx=0x7f213401c890 comp rx=0 tx=0).stop 2024-04-25T11:19:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f214397d640 1 -- 172.21.15.106:0/2318697012 shutdown_connections 2024-04-25T11:19:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f214397d640 1 --2- 172.21.15.106:0/2318697012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c0cf990 0x7f213c0cfda0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f214397d640 1 --2- 172.21.15.106:0/2318697012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c06cd00 0x7f213c0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f214397d640 1 -- 172.21.15.106:0/2318697012 >> 172.21.15.106:0/2318697012 conn(0x7f213c0b78b0 msgr2=0x7f213c0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f214397d640 1 -- 172.21.15.106:0/2318697012 shutdown_connections 2024-04-25T11:19:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.936+0000 7f214397d640 1 -- 172.21.15.106:0/2318697012 wait complete. 2024-04-25T11:19:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 Processor -- start 2024-04-25T11:19:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 -- start start 2024-04-25T11:19:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c0cf990 0x7f213c0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c15dd50 0x7f213c1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c1fcba0 0x7f213c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f213c16a960 con 0x7f213c15dd50 2024-04-25T11:19:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f213c0b4d60 con 0x7f213c1fcba0 2024-04-25T11:19:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f213c16aae0 con 0x7f213c0cf990 2024-04-25T11:19:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c15dd50 0x7f213c1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c15dd50 0x7f213c1518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45614/0 (socket says 172.21.15.106:45614) 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 -- 172.21.15.106:0/3757496641 learned_addr learned my addr 172.21.15.106:0/3757496641 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2141719640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c0cf990 0x7f213c0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2141f1a640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c1fcba0 0x7f213c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 -- 172.21.15.106:0/3757496641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c0cf990 msgr2=0x7f213c0b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c0cf990 0x7f213c0b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 -- 172.21.15.106:0/3757496641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c1fcba0 msgr2=0x7f213c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c1fcba0 0x7f213c15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 -- 172.21.15.106:0/3757496641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c0b5af0 con 0x7f213c15dd50 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2140f18640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c15dd50 0x7f213c1518d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f212c00c7f0 tx=0x7f212c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2131ffb640 1 -- 172.21.15.106:0/3757496641 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c015070 con 0x7f213c15dd50 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2131ffb640 1 -- 172.21.15.106:0/3757496641 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212c007820 con 0x7f213c15dd50 2024-04-25T11:19:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c0b5dd0 con 0x7f213c15dd50 2024-04-25T11:19:59.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f2131ffb640 1 -- 172.21.15.106:0/3757496641 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c007c10 con 0x7f213c15dd50 2024-04-25T11:19:59.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.940+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c0b6390 con 0x7f213c15dd50 2024-04-25T11:19:59.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.944+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c1615d0 con 0x7f213c15dd50 2024-04-25T11:19:59.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.944+0000 7f2131ffb640 1 -- 172.21.15.106:0/3757496641 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f212c018070 con 0x7f213c15dd50 2024-04-25T11:19:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.944+0000 7f2131ffb640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f212006f400 0x7f2120071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.944+0000 7f2141719640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f212006f400 0x7f2120071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.944+0000 7f2131ffb640 1 -- 172.21.15.106:0/3757496641 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f212c08f410 con 0x7f213c15dd50 2024-04-25T11:19:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.944+0000 7f2141719640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f212006f400 0x7f2120071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2138000f70 tx=0x7f2138009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:19:59.948+0000 7f2131ffb640 1 -- 172.21.15.106:0/3757496641 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f212c05c2f0 con 0x7f213c15dd50 2024-04-25T11:20:00.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.100+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f213c0b5880 con 0x7f213c15dd50 2024-04-25T11:20:00.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f2131ffb640 1 -- 172.21.15.106:0/3757496641 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f212c05bbf0 con 0x7f213c15dd50 2024-04-25T11:20:00.102 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:00.102 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:00.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f212006f400 msgr2=0x7f2120071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f212006f400 0x7f2120071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2138000f70 tx=0x7f2138009040 comp rx=0 tx=0).stop 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c15dd50 msgr2=0x7f213c1518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c15dd50 0x7f213c1518d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f212c00c7f0 tx=0x7f212c00ccb0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 shutdown_connections 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c1fcba0 0x7f213c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f212006f400 0x7f2120071910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f213c15dd50 0x7f213c1518d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 --2- 172.21.15.106:0/3757496641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c0cf990 0x7f213c0b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 >> 172.21.15.106:0/3757496641 conn(0x7f213c0b78b0 msgr2=0x7f213c158070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 shutdown_connections 2024-04-25T11:20:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.104+0000 7f214397d640 1 -- 172.21.15.106:0/3757496641 wait complete. 2024-04-25T11:20:00.123 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:00.123 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ee9e0640 1 Processor -- start 2024-04-25T11:20:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ee9e0640 1 -- start start 2024-04-25T11:20:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ee9e0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 0x7f14e80d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ee9e0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f14e80b74e0 0x7f14e8168240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ee9e0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14e80d1520 con 0x7f14e80d2380 2024-04-25T11:20:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ee9e0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f14e80d1aa0 con 0x7f14e80b74e0 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ee9e0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f14e80cf370 con 0x7f14e809ca40 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e77fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f14e80d2380 0x7f14e80b6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42074/0 (socket says 172.21.15.106:42074) 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e77fe640 1 -- 172.21.15.106:0/476033529 learned_addr learned my addr 172.21.15.106:0/476033529 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e7fff640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 0x7f14e80d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ecf7d640 1 --2- 172.21.15.106:0/476033529 >> v2:172.21.15.134:3300/0 conn(0x7f14e80b74e0 0x7f14e8168240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1814680469 0 0) 0x7f14e80d1520 con 0x7f14e80d2380 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14d4003530 con 0x7f14e80d2380 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2188793761 0 0) 0x7f14d4003530 con 0x7f14e80d2380 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 msgr2=0x7f14e80d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 0x7f14e80d1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 >> v2:172.21.15.134:3300/0 conn(0x7f14e80b74e0 msgr2=0x7f14e8168240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 --2- 172.21.15.106:0/476033529 >> v2:172.21.15.134:3300/0 conn(0x7f14e80b74e0 0x7f14e8168240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e80b8d80 con 0x7f14e80d2380 2024-04-25T11:20:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f14d80016a0 con 0x7f14e80d2380 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 >> v1:172.21.15.106:6789/0 conn(0x7f14e80d2380 legacy=0x7f14e80b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 0x7f14d40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14d4004d50 0x7f14d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14e80d1aa0 con 0x7f14d4003f40 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14e80cf370 con 0x7f14d4004d50 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f14d400bad0 con 0x7f14e809ca40 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e77fe640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e7fff640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 0x7f14d40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ecf7d640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14d4004d50 0x7f14d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e77fe640 1 -- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 msgr2=0x7f14d40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e77fe640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 0x7f14d40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e77fe640 1 -- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14d4004d50 msgr2=0x7f14d4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e77fe640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14d4004d50 0x7f14d4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e77fe640 1 -- 172.21.15.106:0/476033529 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d4003530 con 0x7f14d4003f40 2024-04-25T11:20:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14e7fff640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e809ca40 0x7f14d40039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.264+0000 7f14ecf7d640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14d4004d50 0x7f14d4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14e77fe640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14d4004810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c210 tx=0x7f14d801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc46f5ca3c4d80f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8007d80 con 0x7f14d4003f40 2024-04-25T11:20:00.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d80a9020 con 0x7f14d4003f40 2024-04-25T11:20:00.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14e6ffd640 1 -- 172.21.15.106:0/476033529 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8005ae0 con 0x7f14d4003f40 2024-04-25T11:20:00.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14ee9e0640 1 -- 172.21.15.106:0/476033529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 msgr2=0x7f14d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14ee9e0640 1 --2- 172.21.15.106:0/476033529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14d4004810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c210 tx=0x7f14d801cca0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14ecf7d640 1 -- 172.21.15.106:0/476033529 reap_dead start 2024-04-25T11:20:00.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14ee9e0640 1 -- 172.21.15.106:0/476033529 shutdown_connections 2024-04-25T11:20:00.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14ee9e0640 1 -- 172.21.15.106:0/476033529 >> 172.21.15.106:0/476033529 conn(0x7f14e80cfff0 msgr2=0x7f14e80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14ee9e0640 1 -- 172.21.15.106:0/476033529 shutdown_connections 2024-04-25T11:20:00.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.268+0000 7f14ee9e0640 1 -- 172.21.15.106:0/476033529 wait complete. 2024-04-25T11:20:00.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 Processor -- start 2024-04-25T11:20:00.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 -- start start 2024-04-25T11:20:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14e8154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e815d340 0x7f14e809e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e815b740 0x7f14e809ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14e80c6ea0 con 0x7f14d4003f40 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14e80c6d20 con 0x7f14e815d340 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f14e80b88a0 con 0x7f14e815b740 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14e8154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14e8154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45638/0 (socket says 172.21.15.106:45638) 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 -- 172.21.15.106:0/3268581277 learned_addr learned my addr 172.21.15.106:0/3268581277 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e77fe640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e815d340 0x7f14e809e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ecf7d640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e815b740 0x7f14e809ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 -- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e815b740 msgr2=0x7f14e809ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e815b740 0x7f14e809ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 -- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e815d340 msgr2=0x7f14e809e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e815d340 0x7f14e809e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 -- 172.21.15.106:0/3268581277 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e8155410 con 0x7f14d4003f40 2024-04-25T11:20:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ecf7d640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e815b740 0x7f14e809ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e7fff640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14e8154b10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f14d0010890 tx=0x7f14d0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e4ff9640 1 -- 172.21.15.106:0/3268581277 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d000f040 con 0x7f14d4003f40 2024-04-25T11:20:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e4ff9640 1 -- 172.21.15.106:0/3268581277 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d000eb60 con 0x7f14d4003f40 2024-04-25T11:20:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e815da20 con 0x7f14d4003f40 2024-04-25T11:20:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14e4ff9640 1 -- 172.21.15.106:0/3268581277 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d001c400 con 0x7f14d4003f40 2024-04-25T11:20:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e806da80 con 0x7f14d4003f40 2024-04-25T11:20:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.272+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e8050750 con 0x7f14d4003f40 2024-04-25T11:20:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.276+0000 7f14e4ff9640 1 -- 172.21.15.106:0/3268581277 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14d001c5a0 con 0x7f14d4003f40 2024-04-25T11:20:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.276+0000 7f14e4ff9640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f14c806f400 0x7f14c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.276+0000 7f14e4ff9640 1 -- 172.21.15.106:0/3268581277 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f14d008fa00 con 0x7f14d4003f40 2024-04-25T11:20:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.276+0000 7f14e77fe640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f14c806f400 0x7f14c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.276+0000 7f14e77fe640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f14c806f400 0x7f14c8071910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14d80aa000 tx=0x7f14d8001ae0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.284+0000 7f14e4ff9640 1 -- 172.21.15.106:0/3268581277 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14d0018030 con 0x7f14d4003f40 2024-04-25T11:20:00.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14e80d0610 con 0x7f14d4003f40 2024-04-25T11:20:00.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14e4ff9640 1 -- 172.21.15.106:0/3268581277 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f14d005c870 con 0x7f14d4003f40 2024-04-25T11:20:00.436 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:00.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f14c806f400 msgr2=0x7f14c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f14c806f400 0x7f14c8071910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14d80aa000 tx=0x7f14d8001ae0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 msgr2=0x7f14e8154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14e8154b10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f14d0010890 tx=0x7f14d0010d50 comp rx=0 tx=0).stop 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 shutdown_connections 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14e815b740 0x7f14e809ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e815d340 0x7f14e809e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f14c806f400 0x7f14c8071910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 --2- 172.21.15.106:0/3268581277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d4003f40 0x7f14e8154b10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 >> 172.21.15.106:0/3268581277 conn(0x7f14e80cfff0 msgr2=0x7f14e809c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 shutdown_connections 2024-04-25T11:20:00.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.436+0000 7f14ee9e0640 1 -- 172.21.15.106:0/3268581277 wait complete. 2024-04-25T11:20:00.456 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:00.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d5c785640 1 Processor -- start 2024-04-25T11:20:00.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d5c785640 1 -- start start 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d5c785640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d5406d600 0x7f4d5406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d5c785640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 0x7f4d5415e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d5c785640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5406df50 con 0x7f4d540b6660 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d5c785640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4d540b5380 con 0x7f4d5406d600 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d5c785640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540b6430 con 0x7f4d5415dcf0 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d59d20640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d5406d600 0x7f4d5406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d59d20640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d5406d600 0x7f4d5406da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45644/0 (socket says 172.21.15.106:45644) 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d59d20640 1 -- 172.21.15.106:0/3781910065 learned_addr learned my addr 172.21.15.106:0/3781910065 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d5ad22640 1 --2- 172.21.15.106:0/3781910065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 0x7f4d5415e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d59d20640 1 -- 172.21.15.106:0/3781910065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 msgr2=0x7f4d5415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d59d20640 1 --2- 172.21.15.106:0/3781910065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 0x7f4d5415e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d59d20640 1 -- 172.21.15.106:0/3781910065 >> v1:172.21.15.134:6789/0 conn(0x7f4d540b6660 legacy=0x7f4d540b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.616+0000 7f4d59d20640 1 -- 172.21.15.106:0/3781910065 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d54073fc0 con 0x7f4d5406d600 2024-04-25T11:20:00.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d59d20640 1 --2- 172.21.15.106:0/3781910065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d5406da10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009870 tx=0x7f4d4801c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902f1aaec847a307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5951f640 1 -- 172.21.15.106:0/3781910065 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d480b0070 con 0x7f4d5406d600 2024-04-25T11:20:00.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5951f640 1 -- 172.21.15.106:0/3781910065 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d480ab600 con 0x7f4d5406d600 2024-04-25T11:20:00.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5951f640 1 -- 172.21.15.106:0/3781910065 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d480ab9d0 con 0x7f4d5406d600 2024-04-25T11:20:00.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 -- 172.21.15.106:0/3781910065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 msgr2=0x7f4d5406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 --2- 172.21.15.106:0/3781910065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d5406da10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009870 tx=0x7f4d4801c220 comp rx=0 tx=0).stop 2024-04-25T11:20:00.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 -- 172.21.15.106:0/3781910065 shutdown_connections 2024-04-25T11:20:00.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 --2- 172.21.15.106:0/3781910065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 0x7f4d5415e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 --2- 172.21.15.106:0/3781910065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d5406da10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 -- 172.21.15.106:0/3781910065 >> 172.21.15.106:0/3781910065 conn(0x7f4d540732c0 msgr2=0x7f4d540736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 -- 172.21.15.106:0/3781910065 shutdown_connections 2024-04-25T11:20:00.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 -- 172.21.15.106:0/3781910065 wait complete. 2024-04-25T11:20:00.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.620+0000 7f4d5c785640 1 Processor -- start 2024-04-25T11:20:00.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 -- start start 2024-04-25T11:20:00.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d54149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d540b6660 0x7f4d541f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 0x7f4d5414b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5415ebd0 con 0x7f4d5406d600 2024-04-25T11:20:00.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5415ea50 con 0x7f4d540b6660 2024-04-25T11:20:00.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d5416b070 con 0x7f4d5415dcf0 2024-04-25T11:20:00.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d54149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d54149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45652/0 (socket says 172.21.15.106:45652) 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 -- 172.21.15.106:0/283728714 learned_addr learned my addr 172.21.15.106:0/283728714 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d59d20640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d540b6660 0x7f4d541f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 -- 172.21.15.106:0/283728714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 msgr2=0x7f4d5414b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 0x7f4d5414b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 -- 172.21.15.106:0/283728714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d540b6660 msgr2=0x7f4d541f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d540b6660 0x7f4d541f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 -- 172.21.15.106:0/283728714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d54149570 con 0x7f4d5406d600 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5a521640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d54149030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400ba50 tx=0x7f4d4400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d52ffd640 1 -- 172.21.15.106:0/283728714 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44019070 con 0x7f4d5406d600 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d52ffd640 1 -- 172.21.15.106:0/283728714 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4400cbc0 con 0x7f4d5406d600 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5414a1e0 con 0x7f4d5406d600 2024-04-25T11:20:00.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d52ffd640 1 -- 172.21.15.106:0/283728714 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44014440 con 0x7f4d5406d600 2024-04-25T11:20:00.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5414a7a0 con 0x7f4d5406d600 2024-04-25T11:20:00.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d54050750 con 0x7f4d5406d600 2024-04-25T11:20:00.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.624+0000 7f4d52ffd640 1 -- 172.21.15.106:0/283728714 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4d4401c070 con 0x7f4d5406d600 2024-04-25T11:20:00.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.628+0000 7f4d52ffd640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4d4006f430 0x7f4d40071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.628+0000 7f4d59d20640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4d4006f430 0x7f4d40071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.628+0000 7f4d52ffd640 1 -- 172.21.15.106:0/283728714 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d4408eeb0 con 0x7f4d5406d600 2024-04-25T11:20:00.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.628+0000 7f4d59d20640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4d4006f430 0x7f4d40071940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4d480ad000 tx=0x7f4d48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.632+0000 7f4d52ffd640 1 -- 172.21.15.106:0/283728714 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4d4405b7e0 con 0x7f4d5406d600 2024-04-25T11:20:00.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.792+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d540b6050 con 0x7f4d5406d600 2024-04-25T11:20:00.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.792+0000 7f4d52ffd640 1 -- 172.21.15.106:0/283728714 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f4d4405fae0 con 0x7f4d5406d600 2024-04-25T11:20:00.793 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4d4006f430 msgr2=0x7f4d40071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4d4006f430 0x7f4d40071940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4d480ad000 tx=0x7f4d48008040 comp rx=0 tx=0).stop 2024-04-25T11:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 msgr2=0x7f4d54149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d54149030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400ba50 tx=0x7f4d4400bf10 comp rx=0 tx=0).stop 2024-04-25T11:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 shutdown_connections 2024-04-25T11:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d5415dcf0 0x7f4d5414b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d540b6660 0x7f4d541f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4d4006f430 0x7f4d40071940 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 --2- 172.21.15.106:0/283728714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d5406d600 0x7f4d54149030 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 >> 172.21.15.106:0/283728714 conn(0x7f4d540732c0 msgr2=0x7f4d5415adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 shutdown_connections 2024-04-25T11:20:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.796+0000 7f4d5c785640 1 -- 172.21.15.106:0/283728714 wait complete. 2024-04-25T11:20:00.815 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:00.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cb200c640 1 Processor -- start 2024-04-25T11:20:00.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cb200c640 1 -- start start 2024-04-25T11:20:00.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cb200c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2cac143fa0 0x7f2cac0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cb200c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2cac0d1e50 0x7f2cac0b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cb200c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cac073300 con 0x7f2cac072a20 2024-04-25T11:20:00.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cb200c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2cac0735e0 con 0x7f2cac0d1e50 2024-04-25T11:20:00.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cb200c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2cac0738c0 con 0x7f2cac143fa0 2024-04-25T11:20:00.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cabfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2cac0d1e50 0x7f2cac0b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2caaffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2cac072a20 0x7f2cac0d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42086/0 (socket says 172.21.15.106:42086) 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2caaffd640 1 -- 172.21.15.106:0/2246355866 learned_addr learned my addr 172.21.15.106:0/2246355866 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cab7fe640 1 --2- 172.21.15.106:0/2246355866 >> v2:172.21.15.134:3300/0 conn(0x7f2cac143fa0 0x7f2cac0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cabfff640 1 -- 172.21.15.106:0/2246355866 >> v2:172.21.15.134:3300/0 conn(0x7f2cac143fa0 msgr2=0x7f2cac0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cabfff640 1 --2- 172.21.15.106:0/2246355866 >> v2:172.21.15.134:3300/0 conn(0x7f2cac143fa0 0x7f2cac0cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cabfff640 1 -- 172.21.15.106:0/2246355866 >> v1:172.21.15.106:6789/0 conn(0x7f2cac072a20 legacy=0x7f2cac0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cabfff640 1 -- 172.21.15.106:0/2246355866 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac0b5040 con 0x7f2cac0d1e50 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2caa7fc640 1 -- 172.21.15.106:0/2246355866 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 888152125 0 0) 0x7f2cac073300 con 0x7f2cac072a20 2024-04-25T11:20:00.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.968+0000 7f2cab7fe640 1 --2- 172.21.15.106:0/2246355866 >> v2:172.21.15.134:3300/0 conn(0x7f2cac143fa0 0x7f2cac0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:00.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cabfff640 1 --2- 172.21.15.106:0/2246355866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac0d1e50 0x7f2cac0b5840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2cac070f30 tx=0x7f2ca401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3cc626a2fa5c9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2caa7fc640 1 -- 172.21.15.106:0/2246355866 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40b2070 con 0x7f2cac0d1e50 2024-04-25T11:20:00.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2caa7fc640 1 -- 172.21.15.106:0/2246355866 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca40a59e0 con 0x7f2cac0d1e50 2024-04-25T11:20:00.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2caa7fc640 1 -- 172.21.15.106:0/2246355866 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40a5db0 con 0x7f2cac0d1e50 2024-04-25T11:20:00.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 -- 172.21.15.106:0/2246355866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac0d1e50 msgr2=0x7f2cac0b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 --2- 172.21.15.106:0/2246355866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac0d1e50 0x7f2cac0b5840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2cac070f30 tx=0x7f2ca401c900 comp rx=0 tx=0).stop 2024-04-25T11:20:00.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 -- 172.21.15.106:0/2246355866 shutdown_connections 2024-04-25T11:20:00.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 --2- 172.21.15.106:0/2246355866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac0d1e50 0x7f2cac0b5840 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 --2- 172.21.15.106:0/2246355866 >> v2:172.21.15.134:3300/0 conn(0x7f2cac143fa0 0x7f2cac0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 -- 172.21.15.106:0/2246355866 >> 172.21.15.106:0/2246355866 conn(0x7f2cac06c9f0 msgr2=0x7f2cac06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 -- 172.21.15.106:0/2246355866 shutdown_connections 2024-04-25T11:20:00.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 -- 172.21.15.106:0/2246355866 wait complete. 2024-04-25T11:20:00.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 Processor -- start 2024-04-25T11:20:00.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.972+0000 7f2cb200c640 1 -- start start 2024-04-25T11:20:00.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cac072a20 0x7f2cac13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac143fa0 0x7f2cac14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac14de70 0x7f2cac15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac16ad20 con 0x7f2cac143fa0 2024-04-25T11:20:00.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac16aba0 con 0x7f2cac14de70 2024-04-25T11:20:00.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2cac16aea0 con 0x7f2cac072a20 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac143fa0 0x7f2cac14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac143fa0 0x7f2cac14d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45678/0 (socket says 172.21.15.106:45678) 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 -- 172.21.15.106:0/821245523 learned_addr learned my addr 172.21.15.106:0/821245523 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cabfff640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac14de70 0x7f2cac15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 -- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cac072a20 msgr2=0x7f2cac13fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cab7fe640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cac072a20 0x7f2cac13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cac072a20 0x7f2cac13fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 -- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac14de70 msgr2=0x7f2cac15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac14de70 0x7f2cac15cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 -- 172.21.15.106:0/821245523 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac14e280 con 0x7f2cac143fa0 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cabfff640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac14de70 0x7f2cac15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2caaffd640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac143fa0 0x7f2cac14d930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000c480 tx=0x7f2ca000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2c93fff640 1 -- 172.21.15.106:0/821245523 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0018070 con 0x7f2cac143fa0 2024-04-25T11:20:00.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2c93fff640 1 -- 172.21.15.106:0/821245523 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca0007500 con 0x7f2cac143fa0 2024-04-25T11:20:00.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac15bd40 con 0x7f2cac143fa0 2024-04-25T11:20:00.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2c93fff640 1 -- 172.21.15.106:0/821245523 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0004a30 con 0x7f2cac143fa0 2024-04-25T11:20:00.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac15c2c0 con 0x7f2cac143fa0 2024-04-25T11:20:00.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac050750 con 0x7f2cac143fa0 2024-04-25T11:20:00.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2c93fff640 1 -- 172.21.15.106:0/821245523 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ca001b070 con 0x7f2cac143fa0 2024-04-25T11:20:00.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.976+0000 7f2c93fff640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2c9806f400 0x7f2c98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.980+0000 7f2cab7fe640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2c9806f400 0x7f2c98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.980+0000 7f2c93fff640 1 -- 172.21.15.106:0/821245523 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ca008e860 con 0x7f2cac143fa0 2024-04-25T11:20:00.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.980+0000 7f2cab7fe640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2c9806f400 0x7f2c98071910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0096d0 tx=0x7f2c9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:00.980+0000 7f2c93fff640 1 -- 172.21.15.106:0/821245523 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2cac14e280 con 0x7f2cac143fa0 2024-04-25T11:20:01.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.128+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2cac0b4e20 con 0x7f2cac143fa0 2024-04-25T11:20:01.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.128+0000 7f2c93fff640 1 -- 172.21.15.106:0/821245523 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f2ca005b8c0 con 0x7f2cac143fa0 2024-04-25T11:20:01.129 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2c9806f400 msgr2=0x7f2c98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2c9806f400 0x7f2c98071910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0096d0 tx=0x7f2c9c008040 comp rx=0 tx=0).stop 2024-04-25T11:20:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac143fa0 msgr2=0x7f2cac14d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac143fa0 0x7f2cac14d930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000c480 tx=0x7f2ca000c940 comp rx=0 tx=0).stop 2024-04-25T11:20:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 shutdown_connections 2024-04-25T11:20:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac14de70 0x7f2cac15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2c9806f400 0x7f2c98071910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac143fa0 0x7f2cac14d930 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 --2- 172.21.15.106:0/821245523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cac072a20 0x7f2cac13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 >> 172.21.15.106:0/821245523 conn(0x7f2cac06c9f0 msgr2=0x7f2cac15a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 shutdown_connections 2024-04-25T11:20:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.132+0000 7f2cb200c640 1 -- 172.21.15.106:0/821245523 wait complete. 2024-04-25T11:20:01.149 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_readahead.TestReadahead.test_flush' 2024-04-25T11:20:01.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.284+0000 7fede4647640 1 Processor -- start 2024-04-25T11:20:01.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.284+0000 7fede4647640 1 -- start start 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede4647640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feddc156f90 0x7feddc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede4647640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7feddc0b5a80 0x7feddc0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede4647640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feddc1573a0 con 0x7feddc06fea0 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede4647640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feddc0d1f90 con 0x7feddc156f90 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede4647640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7feddc0b4810 con 0x7feddc0b5a80 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede23e3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feddc156f90 0x7feddc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede23e3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feddc156f90 0x7feddc0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45692/0 (socket says 172.21.15.106:45692) 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede23e3640 1 -- 172.21.15.106:0/1208091371 learned_addr learned my addr 172.21.15.106:0/1208091371 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede2be4640 1 --2- 172.21.15.106:0/1208091371 >> v2:172.21.15.134:3300/0 conn(0x7feddc0b5a80 0x7feddc0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede23e3640 1 -- 172.21.15.106:0/1208091371 >> v2:172.21.15.134:3300/0 conn(0x7feddc0b5a80 msgr2=0x7feddc0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:01.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede23e3640 1 --2- 172.21.15.106:0/1208091371 >> v2:172.21.15.134:3300/0 conn(0x7feddc0b5a80 0x7feddc0b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede23e3640 1 -- 172.21.15.106:0/1208091371 >> v1:172.21.15.134:6789/0 conn(0x7feddc06fea0 legacy=0x7feddc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:01.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede23e3640 1 -- 172.21.15.106:0/1208091371 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc15d220 con 0x7feddc156f90 2024-04-25T11:20:01.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede23e3640 1 --2- 172.21.15.106:0/1208091371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc0d1a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feddc0b5540 tx=0x7fedd401eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5668c5b29b0ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede13e1640 1 -- 172.21.15.106:0/1208091371 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40b4070 con 0x7feddc156f90 2024-04-25T11:20:01.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede13e1640 1 -- 172.21.15.106:0/1208091371 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd40a79d0 con 0x7feddc156f90 2024-04-25T11:20:01.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.288+0000 7fede13e1640 1 -- 172.21.15.106:0/1208091371 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40a7da0 con 0x7feddc156f90 2024-04-25T11:20:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- 172.21.15.106:0/1208091371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 msgr2=0x7feddc0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 --2- 172.21.15.106:0/1208091371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc0d1a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feddc0b5540 tx=0x7fedd401eaf0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- 172.21.15.106:0/1208091371 shutdown_connections 2024-04-25T11:20:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 --2- 172.21.15.106:0/1208091371 >> v2:172.21.15.134:3300/0 conn(0x7feddc0b5a80 0x7feddc0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 --2- 172.21.15.106:0/1208091371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc0d1a50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- 172.21.15.106:0/1208091371 >> 172.21.15.106:0/1208091371 conn(0x7feddc06c7c0 msgr2=0x7feddc06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- 172.21.15.106:0/1208091371 shutdown_connections 2024-04-25T11:20:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- 172.21.15.106:0/1208091371 wait complete. 2024-04-25T11:20:01.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 Processor -- start 2024-04-25T11:20:01.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- start start 2024-04-25T11:20:01.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feddc06fea0 0x7feddc164450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feddc0b5a80 0x7feddc153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc165c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feddc15dda0 con 0x7feddc156f90 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feddc15ced0 con 0x7feddc0b5a80 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede4647640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feddc0d1290 con 0x7feddc06fea0 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede2be4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc165c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede2be4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc165c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45702/0 (socket says 172.21.15.106:45702) 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede2be4640 1 -- 172.21.15.106:0/1723624571 learned_addr learned my addr 172.21.15.106:0/1723624571 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede1be2640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feddc0b5a80 0x7feddc153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede23e3640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feddc06fea0 0x7feddc164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede2be4640 1 -- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feddc06fea0 msgr2=0x7feddc164450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede2be4640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feddc06fea0 0x7feddc164450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede2be4640 1 -- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feddc0b5a80 msgr2=0x7feddc153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede2be4640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feddc0b5a80 0x7feddc153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.292+0000 7fede2be4640 1 -- 172.21.15.106:0/1723624571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc1538c0 con 0x7feddc156f90 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fede1be2640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feddc0b5a80 0x7feddc153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fede2be4640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc165c70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009fd0 tx=0x7fedd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fedd2ffd640 1 -- 172.21.15.106:0/1723624571 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8014420 con 0x7feddc156f90 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fedd2ffd640 1 -- 172.21.15.106:0/1723624571 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8019070 con 0x7feddc156f90 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc1633b0 con 0x7feddc156f90 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fedd2ffd640 1 -- 172.21.15.106:0/1723624571 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8014800 con 0x7feddc156f90 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc163970 con 0x7feddc156f90 2024-04-25T11:20:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feddc06f760 con 0x7feddc156f90 2024-04-25T11:20:01.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fedd2ffd640 1 -- 172.21.15.106:0/1723624571 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fedd80149a0 con 0x7feddc156f90 2024-04-25T11:20:01.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fedd2ffd640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fedc006f400 0x7fedc0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fede23e3640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fedc006f400 0x7fedc0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fedd2ffd640 1 -- 172.21.15.106:0/1723624571 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fedd808fb10 con 0x7feddc156f90 2024-04-25T11:20:01.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.296+0000 7fede23e3640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fedc006f400 0x7fedc0071910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fedd40b1000 tx=0x7fedd40a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.300+0000 7fedd2ffd640 1 -- 172.21.15.106:0/1723624571 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fedd805c3b0 con 0x7feddc156f90 2024-04-25T11:20:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:01.512+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_readahead.TestReadahead.test_flush"]} v 0) -- 0x7feddc163100 con 0x7feddc156f90 2024-04-25T11:20:02.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.212+0000 7fedd2ffd640 1 -- 172.21.15.106:0/1723624571 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_readahead.TestReadahead.test_flush"]}]=0 v56) ==== 134+0+0 (secure 0 0 0) 0x7fedd80606b0 con 0x7feddc156f90 2024-04-25T11:20:02.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fedc006f400 msgr2=0x7fedc0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fedc006f400 0x7fedc0071910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fedd40b1000 tx=0x7fedd40a9070 comp rx=0 tx=0).stop 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 msgr2=0x7feddc165c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc165c70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009fd0 tx=0x7fedd800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 shutdown_connections 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fedc006f400 0x7fedc0071910 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feddc156f90 0x7feddc165c70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feddc0b5a80 0x7feddc153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 --2- 172.21.15.106:0/1723624571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feddc06fea0 0x7feddc164450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 >> 172.21.15.106:0/1723624571 conn(0x7feddc06c7c0 msgr2=0x7feddc166eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 shutdown_connections 2024-04-25T11:20:02.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.216+0000 7fede4647640 1 -- 172.21.15.106:0/1723624571 wait complete. 2024-04-25T11:20:02.239 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:20:02.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.380+0000 7f1e71374640 1 Processor -- start 2024-04-25T11:20:02.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.380+0000 7f1e71374640 1 -- start start 2024-04-25T11:20:02.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e71374640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1e6c0d0cf0 0x7f1e6c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e71374640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c15d230 0x7f1e6c15d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e71374640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6c0b68e0 con 0x7f1e6c0d1740 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e71374640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6c161210 con 0x7f1e6c0d0cf0 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e71374640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c0b6290 con 0x7f1e6c15d230 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e6affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1e6c0d0cf0 0x7f1e6c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e6a7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1e6c0d1740 0x7f1e6c0d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42090/0 (socket says 172.21.15.106:42090) 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e6a7fc640 1 -- 172.21.15.106:0/873901446 learned_addr learned my addr 172.21.15.106:0/873901446 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e6affd640 1 -- 172.21.15.106:0/873901446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c15d230 msgr2=0x7f1e6c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e6b7fe640 1 --2- 172.21.15.106:0/873901446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c15d230 0x7f1e6c15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e6affd640 1 --2- 172.21.15.106:0/873901446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c15d230 0x7f1e6c15d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e6affd640 1 -- 172.21.15.106:0/873901446 >> v1:172.21.15.106:6789/0 conn(0x7f1e6c0d1740 legacy=0x7f1e6c0d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e69ffb640 1 -- 172.21.15.106:0/873901446 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784592198 0 0) 0x7f1e6c0b68e0 con 0x7f1e6c0d1740 2024-04-25T11:20:02.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.384+0000 7f1e6affd640 1 -- 172.21.15.106:0/873901446 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c0b6570 con 0x7f1e6c0d0cf0 2024-04-25T11:20:02.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e6affd640 1 --2- 172.21.15.106:0/873901446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d0cf0 0x7f1e6c06ea20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0097c0 tx=0x7f1e5c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d102a0cec3190b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e69ffb640 1 -- 172.21.15.106:0/873901446 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0b0070 con 0x7f1e6c0d0cf0 2024-04-25T11:20:02.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e69ffb640 1 -- 172.21.15.106:0/873901446 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c0ab600 con 0x7f1e6c0d0cf0 2024-04-25T11:20:02.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e69ffb640 1 -- 172.21.15.106:0/873901446 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0ab9d0 con 0x7f1e6c0d0cf0 2024-04-25T11:20:02.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 -- 172.21.15.106:0/873901446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d0cf0 msgr2=0x7f1e6c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 --2- 172.21.15.106:0/873901446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d0cf0 0x7f1e6c06ea20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0097c0 tx=0x7f1e5c01c380 comp rx=0 tx=0).stop 2024-04-25T11:20:02.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 -- 172.21.15.106:0/873901446 shutdown_connections 2024-04-25T11:20:02.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 --2- 172.21.15.106:0/873901446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c15d230 0x7f1e6c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 --2- 172.21.15.106:0/873901446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d0cf0 0x7f1e6c06ea20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 -- 172.21.15.106:0/873901446 >> 172.21.15.106:0/873901446 conn(0x7f1e6c06c9f0 msgr2=0x7f1e6c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:02.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 -- 172.21.15.106:0/873901446 shutdown_connections 2024-04-25T11:20:02.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 -- 172.21.15.106:0/873901446 wait complete. 2024-04-25T11:20:02.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 Processor -- start 2024-04-25T11:20:02.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.388+0000 7f1e71374640 1 -- start start 2024-04-25T11:20:02.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c0d0cf0 0x7f1e6c144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c0d1740 0x7f1e6c153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c15d230 0x7f1e6c1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c16a080 con 0x7f1e6c15d230 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c15cdd0 con 0x7f1e6c0d1740 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c16a200 con 0x7f1e6c0d0cf0 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c15d230 0x7f1e6c1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c15d230 0x7f1e6c1564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45726/0 (socket says 172.21.15.106:45726) 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 -- 172.21.15.106:0/2443629169 learned_addr learned my addr 172.21.15.106:0/2443629169 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6a7fc640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c0d1740 0x7f1e6c153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6affd640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c0d0cf0 0x7f1e6c144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 -- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c0d0cf0 msgr2=0x7f1e6c144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c0d0cf0 0x7f1e6c144de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 -- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c0d1740 msgr2=0x7f1e6c153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c0d1740 0x7f1e6c153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 -- 172.21.15.106:0/2443629169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c153540 con 0x7f1e6c15d230 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6affd640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c0d0cf0 0x7f1e6c144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6b7fe640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c15d230 0x7f1e6c1564e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400cc60 tx=0x7f1e6400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e537fe640 1 -- 172.21.15.106:0/2443629169 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e640039a0 con 0x7f1e6c15d230 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e537fe640 1 -- 172.21.15.106:0/2443629169 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e64003b40 con 0x7f1e6c15d230 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c154af0 con 0x7f1e6c15d230 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e537fe640 1 -- 172.21.15.106:0/2443629169 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64004020 con 0x7f1e6c15d230 2024-04-25T11:20:02.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c1550b0 con 0x7f1e6c15d230 2024-04-25T11:20:02.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c050750 con 0x7f1e6c15d230 2024-04-25T11:20:02.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e537fe640 1 -- 172.21.15.106:0/2443629169 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1e640041c0 con 0x7f1e6c15d230 2024-04-25T11:20:02.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e537fe640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1e5806f400 0x7f1e58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e537fe640 1 -- 172.21.15.106:0/2443629169 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e6408fad0 con 0x7f1e6c15d230 2024-04-25T11:20:02.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.392+0000 7f1e6affd640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1e5806f400 0x7f1e58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.396+0000 7f1e6affd640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1e5806f400 0x7f1e58071910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0ad000 tx=0x7f1e5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.396+0000 7f1e537fe640 1 -- 172.21.15.106:0/2443629169 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e6408fe20 con 0x7f1e6c15d230 2024-04-25T11:20:02.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.548+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1e6c154880 con 0x7f1e6c15d230 2024-04-25T11:20:02.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e537fe640 1 -- 172.21.15.106:0/2443629169 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f1e64014030 con 0x7f1e6c15d230 2024-04-25T11:20:02.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1e5806f400 msgr2=0x7f1e58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1e5806f400 0x7f1e58071910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0ad000 tx=0x7f1e5c008040 comp rx=0 tx=0).stop 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c15d230 msgr2=0x7f1e6c1564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c15d230 0x7f1e6c1564e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400cc60 tx=0x7f1e6400a5c0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 shutdown_connections 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1e5806f400 0x7f1e58071910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c15d230 0x7f1e6c1564e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c0d1740 0x7f1e6c153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 --2- 172.21.15.106:0/2443629169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e6c0d0cf0 0x7f1e6c144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 >> 172.21.15.106:0/2443629169 conn(0x7f1e6c06c9f0 msgr2=0x7f1e6c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 shutdown_connections 2024-04-25T11:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.552+0000 7f1e71374640 1 -- 172.21.15.106:0/2443629169 wait complete. 2024-04-25T11:20:02.572 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:20:02.572 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:20:02.635 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:20:02.677 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:20:02.678 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:02.678 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:02.683 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:02.683 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:02.730 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:02.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38aea9d640 1 Processor -- start 2024-04-25T11:20:02.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38aea9d640 1 -- start start 2024-04-25T11:20:02.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38aea9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38a80ba970 0x7f38a80d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38aea9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80b5120 0x7f38a80b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38aea9d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a80d0530 con 0x7f38a80d2160 2024-04-25T11:20:02.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38aea9d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f38a80d1100 con 0x7f38a80ba970 2024-04-25T11:20:02.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38aea9d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38a80d07d0 con 0x7f38a80b5120 2024-04-25T11:20:02.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38ac839640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38a80ba970 0x7f38a80d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38a7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f38a80d2160 0x7f38a80d2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42104/0 (socket says 172.21.15.106:42104) 2024-04-25T11:20:02.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38a7fff640 1 -- 172.21.15.106:0/1582704844 learned_addr learned my addr 172.21.15.106:0/1582704844 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:02.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38ad03a640 1 --2- 172.21.15.106:0/1582704844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80b5120 0x7f38a80b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38ac839640 1 -- 172.21.15.106:0/1582704844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80b5120 msgr2=0x7f38a80b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:02.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38ac839640 1 --2- 172.21.15.106:0/1582704844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80b5120 0x7f38a80b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38ac839640 1 -- 172.21.15.106:0/1582704844 >> v1:172.21.15.106:6789/0 conn(0x7f38a80d2160 legacy=0x7f38a80d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38ac839640 1 -- 172.21.15.106:0/1582704844 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a80d0ab0 con 0x7f38a80ba970 2024-04-25T11:20:02.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.932+0000 7f38ad03a640 1 --2- 172.21.15.106:0/1582704844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80b5120 0x7f38a80b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:02.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38ac839640 1 --2- 172.21.15.106:0/1582704844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80ba970 0x7f38a80d1590 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f38980097a0 tx=0x7f389801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d0231c5222c539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38a77fe640 1 -- 172.21.15.106:0/1582704844 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980b0070 con 0x7f38a80ba970 2024-04-25T11:20:02.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38a77fe640 1 -- 172.21.15.106:0/1582704844 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38980ab600 con 0x7f38a80ba970 2024-04-25T11:20:02.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38a77fe640 1 -- 172.21.15.106:0/1582704844 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980ab9d0 con 0x7f38a80ba970 2024-04-25T11:20:02.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38aea9d640 1 -- 172.21.15.106:0/1582704844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80ba970 msgr2=0x7f38a80d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38aea9d640 1 --2- 172.21.15.106:0/1582704844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80ba970 0x7f38a80d1590 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f38980097a0 tx=0x7f389801c480 comp rx=0 tx=0).stop 2024-04-25T11:20:02.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38aea9d640 1 -- 172.21.15.106:0/1582704844 shutdown_connections 2024-04-25T11:20:02.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38aea9d640 1 --2- 172.21.15.106:0/1582704844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80b5120 0x7f38a80b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38aea9d640 1 --2- 172.21.15.106:0/1582704844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80ba970 0x7f38a80d1590 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38aea9d640 1 -- 172.21.15.106:0/1582704844 >> 172.21.15.106:0/1582704844 conn(0x7f38a80d57e0 msgr2=0x7f38a80d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:02.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38aea9d640 1 -- 172.21.15.106:0/1582704844 shutdown_connections 2024-04-25T11:20:02.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.936+0000 7f38aea9d640 1 -- 172.21.15.106:0/1582704844 wait complete. 2024-04-25T11:20:02.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 Processor -- start 2024-04-25T11:20:02.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 -- start start 2024-04-25T11:20:02.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80b5120 0x7f38a81faae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a80ba970 0x7f38a80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80d2160 0x7f38a80ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80bb200 con 0x7f38a80b5120 2024-04-25T11:20:02.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80bb080 con 0x7f38a80ba970 2024-04-25T11:20:02.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38a80b5a70 con 0x7f38a80d2160 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80b5120 0x7f38a81faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80b5120 0x7f38a81faae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45742/0 (socket says 172.21.15.106:45742) 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 -- 172.21.15.106:0/2172792380 learned_addr learned my addr 172.21.15.106:0/2172792380 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38a7fff640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a80ba970 0x7f38a80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ad03a640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80d2160 0x7f38a80ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 -- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80d2160 msgr2=0x7f38a80ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80d2160 0x7f38a80ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 -- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a80ba970 msgr2=0x7f38a80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a80ba970 0x7f38a80cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 -- 172.21.15.106:0/2172792380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a81fa7d0 con 0x7f38a80b5120 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38a7fff640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a80ba970 0x7f38a80cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:02.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38ac839640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80b5120 0x7f38a81faae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f389801a0b0 tx=0x7f389801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38a57fa640 1 -- 172.21.15.106:0/2172792380 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980b0050 con 0x7f38a80b5120 2024-04-25T11:20:02.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38a57fa640 1 -- 172.21.15.106:0/2172792380 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38980bc070 con 0x7f38a80b5120 2024-04-25T11:20:02.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a81fa4f0 con 0x7f38a80b5120 2024-04-25T11:20:02.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38a57fa640 1 -- 172.21.15.106:0/2172792380 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980b5a10 con 0x7f38a80b5120 2024-04-25T11:20:02.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a80cf3c0 con 0x7f38a80b5120 2024-04-25T11:20:02.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.940+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a815d280 con 0x7f38a80b5120 2024-04-25T11:20:02.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.944+0000 7f38a57fa640 1 -- 172.21.15.106:0/2172792380 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f38980ab4c0 con 0x7f38a80b5120 2024-04-25T11:20:02.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.944+0000 7f38a57fa640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f389406f400 0x7f3894071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.944+0000 7f38a57fa640 1 -- 172.21.15.106:0/2172792380 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3898129970 con 0x7f38a80b5120 2024-04-25T11:20:02.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.944+0000 7f38a7fff640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f389406f400 0x7f3894071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.944+0000 7f38a7fff640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f389406f400 0x7f3894071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f389c0010f0 tx=0x7f389c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:02.948+0000 7f38a57fa640 1 -- 172.21.15.106:0/2172792380 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38980c7060 con 0x7f38a80b5120 2024-04-25T11:20:03.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.108+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38a80d2e90 con 0x7f38a80b5120 2024-04-25T11:20:03.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.112+0000 7f38a57fa640 1 -- 172.21.15.106:0/2172792380 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f3898016070 con 0x7f38a80b5120 2024-04-25T11:20:03.112 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:03.112 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f389406f400 msgr2=0x7f3894071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f389406f400 0x7f3894071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f389c0010f0 tx=0x7f389c009040 comp rx=0 tx=0).stop 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80b5120 msgr2=0x7f38a81faae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80b5120 0x7f38a81faae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f389801a0b0 tx=0x7f389801acc0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 shutdown_connections 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38a80d2160 0x7f38a80ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a80ba970 0x7f38a80cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f389406f400 0x7f3894071910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 --2- 172.21.15.106:0/2172792380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a80b5120 0x7f38a81faae0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 >> 172.21.15.106:0/2172792380 conn(0x7f38a80d57e0 msgr2=0x7f38a806e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 shutdown_connections 2024-04-25T11:20:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.116+0000 7f38aea9d640 1 -- 172.21.15.106:0/2172792380 wait complete. 2024-04-25T11:20:03.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:03.134 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.276+0000 7f53e984f640 1 Processor -- start 2024-04-25T11:20:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e984f640 1 -- start start 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e984f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e41669f0 0x7f53e4168e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e984f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40d0cd0 con 0x7f53e4143f10 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e984f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40d0fa0 con 0x7f53e40d1740 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e984f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53e4144320 con 0x7f53e41669f0 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e2ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f53e4143f10 0x7f53e40d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42110/0 (socket says 172.21.15.106:42110) 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e2ffd640 1 -- 172.21.15.106:0/2403558568 learned_addr learned my addr 172.21.15.106:0/2403558568 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 868969006 0 0) 0x7f53e40d0cd0 con 0x7f53e4143f10 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53cc003530 con 0x7f53e4143f10 2024-04-25T11:20:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e37fe640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e41669f0 0x7f53e4168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2666777485 0 0) 0x7f53cc003530 con 0x7f53e4143f10 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e41669f0 msgr2=0x7f53e4168e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e41669f0 0x7f53e4168e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 >> v1:172.21.15.134:6789/0 conn(0x7f53e40d1740 legacy=0x7f53e40d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e41618c0 con 0x7f53e4143f10 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f53d4001680 con 0x7f53e4143f10 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 >> v1:172.21.15.106:6789/0 conn(0x7f53e4143f10 legacy=0x7f53e40d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc0039f0 0x7f53cc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53cc004810 0x7f53cc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e41669f0 0x7f53cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc0055f0 con 0x7f53cc004810 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc00bf10 con 0x7f53cc0039f0 2024-04-25T11:20:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53cc00c090 con 0x7f53e41669f0 2024-04-25T11:20:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e27fc640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53cc004810 0x7f53cc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc0039f0 0x7f53cc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e27fc640 1 -- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e41669f0 msgr2=0x7f53cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e27fc640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e41669f0 0x7f53cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e27fc640 1 -- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc0039f0 msgr2=0x7f53cc003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e27fc640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc0039f0 0x7f53cc003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e27fc640 1 -- 172.21.15.106:0/2403558568 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc003530 con 0x7f53cc004810 2024-04-25T11:20:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.280+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc0039f0 0x7f53cc003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e27fc640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53cc004810 0x7f53cc004c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f53d8001780 tx=0x7f53d801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ae438232edbda3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80013a0 con 0x7f53cc004810 2024-04-25T11:20:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d80052f0 con 0x7f53cc004810 2024-04-25T11:20:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e1ffb640 1 -- 172.21.15.106:0/2403558568 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8003840 con 0x7f53cc004810 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e984f640 1 -- 172.21.15.106:0/2403558568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53cc004810 msgr2=0x7f53cc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e984f640 1 --2- 172.21.15.106:0/2403558568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53cc004810 0x7f53cc004c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f53d8001780 tx=0x7f53d801c420 comp rx=0 tx=0).stop 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e37fe640 1 -- 172.21.15.106:0/2403558568 reap_dead start 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e984f640 1 -- 172.21.15.106:0/2403558568 shutdown_connections 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e984f640 1 -- 172.21.15.106:0/2403558568 >> 172.21.15.106:0/2403558568 conn(0x7f53e406c9f0 msgr2=0x7f53e406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e984f640 1 -- 172.21.15.106:0/2403558568 shutdown_connections 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e984f640 1 -- 172.21.15.106:0/2403558568 wait complete. 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e984f640 1 Processor -- start 2024-04-25T11:20:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.284+0000 7f53e984f640 1 -- start start 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc004810 0x7f53e40b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e415c820 0x7f53e415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e413dea0 0x7f53e413e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4169620 con 0x7f53e413dea0 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41625f0 con 0x7f53cc004810 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53e41697a0 con 0x7f53e415c820 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e413dea0 0x7f53e413e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e413dea0 0x7f53e413e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45756/0 (socket says 172.21.15.106:45756) 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 -- 172.21.15.106:0/259864451 learned_addr learned my addr 172.21.15.106:0/259864451 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc004810 0x7f53e40b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e27fc640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e415c820 0x7f53e415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 -- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e415c820 msgr2=0x7f53e415c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e415c820 0x7f53e415c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 -- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc004810 msgr2=0x7f53e40b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc004810 0x7f53e40b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 -- 172.21.15.106:0/259864451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e413e7f0 con 0x7f53e413dea0 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e27fc640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e415c820 0x7f53e415c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e37fe640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e413dea0 0x7f53e413e2b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f53e4070c50 tx=0x7f53dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53cbfff640 1 -- 172.21.15.106:0/259864451 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00eb90 con 0x7f53e413dea0 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53cbfff640 1 -- 172.21.15.106:0/259864451 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc00ed50 con 0x7f53e413dea0 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53cbfff640 1 -- 172.21.15.106:0/259864451 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc003be0 con 0x7f53e413dea0 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e4140a10 con 0x7f53e413dea0 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e413eeb0 con 0x7f53e413dea0 2024-04-25T11:20:03.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e415e120 con 0x7f53e413dea0 2024-04-25T11:20:03.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53cbfff640 1 -- 172.21.15.106:0/259864451 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f53dc01b020 con 0x7f53e413dea0 2024-04-25T11:20:03.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.288+0000 7f53cbfff640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f53d006f430 0x7f53d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.292+0000 7f53cbfff640 1 -- 172.21.15.106:0/259864451 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53dc08f180 con 0x7f53e413dea0 2024-04-25T11:20:03.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.292+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f53d006f430 0x7f53d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.292+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f53d006f430 0x7f53d0071940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f53d4001210 tx=0x7f53e415e080 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.296+0000 7f53cbfff640 1 -- 172.21.15.106:0/259864451 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f53e413e7f0 con 0x7f53e413dea0 2024-04-25T11:20:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f53e4050750 con 0x7f53e413dea0 2024-04-25T11:20:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53cbfff640 1 -- 172.21.15.106:0/259864451 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f53dc05c070 con 0x7f53e413dea0 2024-04-25T11:20:03.451 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f53d006f430 msgr2=0x7f53d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f53d006f430 0x7f53d0071940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f53d4001210 tx=0x7f53e415e080 comp rx=0 tx=0).stop 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e413dea0 msgr2=0x7f53e413e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e413dea0 0x7f53e413e2b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f53e4070c50 tx=0x7f53dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 shutdown_connections 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f53d006f430 0x7f53d0071940 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e413dea0 0x7f53e413e2b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53e415c820 0x7f53e415c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 --2- 172.21.15.106:0/259864451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53cc004810 0x7f53e40b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 >> 172.21.15.106:0/259864451 conn(0x7f53e406c9f0 msgr2=0x7f53e4075920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 shutdown_connections 2024-04-25T11:20:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.452+0000 7f53e984f640 1 -- 172.21.15.106:0/259864451 wait complete. 2024-04-25T11:20:03.472 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:03.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1767f94640 1 Processor -- start 2024-04-25T11:20:03.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1767f94640 1 -- start start 2024-04-25T11:20:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1767f94640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f17600b4bd0 0x7f17600ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1767f94640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f176006c5f0 0x7f176006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1767f94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17600b25c0 0x7f17600b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1767f94640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f17600b4fe0 con 0x7f176006c5f0 2024-04-25T11:20:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1767f94640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f176006cf40 con 0x7f17600b4bd0 2024-04-25T11:20:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1767f94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17600b36e0 con 0x7f17600b25c0 2024-04-25T11:20:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f176552f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f176006c5f0 0x7f176006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f176552f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f176006c5f0 0x7f176006ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45764/0 (socket says 172.21.15.106:45764) 2024-04-25T11:20:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f176552f640 1 -- 172.21.15.106:0/2845994936 learned_addr learned my addr 172.21.15.106:0/2845994936 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1765d30640 1 --2- 172.21.15.106:0/2845994936 >> v2:172.21.15.134:3300/0 conn(0x7f17600b4bd0 0x7f17600ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f1766531640 1 --2- 172.21.15.106:0/2845994936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17600b25c0 0x7f17600b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f176552f640 1 -- 172.21.15.106:0/2845994936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17600b25c0 msgr2=0x7f17600b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f176552f640 1 --2- 172.21.15.106:0/2845994936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17600b25c0 0x7f17600b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f176552f640 1 -- 172.21.15.106:0/2845994936 >> v2:172.21.15.134:3300/0 conn(0x7f17600b4bd0 msgr2=0x7f17600ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f176552f640 1 --2- 172.21.15.106:0/2845994936 >> v2:172.21.15.134:3300/0 conn(0x7f17600b4bd0 0x7f17600ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.612+0000 7f176552f640 1 -- 172.21.15.106:0/2845994936 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17600b39c0 con 0x7f176006c5f0 2024-04-25T11:20:03.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f176552f640 1 --2- 172.21.15.106:0/2845994936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f176006c5f0 0x7f176006ca00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f17600d0050 tx=0x7f175401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13e9842a404bd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1764d2e640 1 -- 172.21.15.106:0/2845994936 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17540b0070 con 0x7f176006c5f0 2024-04-25T11:20:03.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1764d2e640 1 -- 172.21.15.106:0/2845994936 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17540ab600 con 0x7f176006c5f0 2024-04-25T11:20:03.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1764d2e640 1 -- 172.21.15.106:0/2845994936 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17540ab9d0 con 0x7f176006c5f0 2024-04-25T11:20:03.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 -- 172.21.15.106:0/2845994936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f176006c5f0 msgr2=0x7f176006ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 --2- 172.21.15.106:0/2845994936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f176006c5f0 0x7f176006ca00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f17600d0050 tx=0x7f175401c520 comp rx=0 tx=0).stop 2024-04-25T11:20:03.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 -- 172.21.15.106:0/2845994936 shutdown_connections 2024-04-25T11:20:03.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 --2- 172.21.15.106:0/2845994936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17600b25c0 0x7f17600b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 --2- 172.21.15.106:0/2845994936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f176006c5f0 0x7f176006ca00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 --2- 172.21.15.106:0/2845994936 >> v2:172.21.15.134:3300/0 conn(0x7f17600b4bd0 0x7f17600ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 -- 172.21.15.106:0/2845994936 >> 172.21.15.106:0/2845994936 conn(0x7f17600b40b0 msgr2=0x7f17600b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 -- 172.21.15.106:0/2845994936 shutdown_connections 2024-04-25T11:20:03.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.616+0000 7f1767f94640 1 -- 172.21.15.106:0/2845994936 wait complete. 2024-04-25T11:20:03.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 Processor -- start 2024-04-25T11:20:03.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 -- start start 2024-04-25T11:20:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f176006c5f0 0x7f17600b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17600b25c0 0x7f17600b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17600b4bd0 0x7f176014a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17600b1b60 con 0x7f17600b4bd0 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17600b2f10 con 0x7f17600b25c0 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17600b1ce0 con 0x7f176006c5f0 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17600b4bd0 0x7f176014a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17600b4bd0 0x7f176014a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45778/0 (socket says 172.21.15.106:45778) 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 -- 172.21.15.106:0/1056621685 learned_addr learned my addr 172.21.15.106:0/1056621685 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f176552f640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17600b25c0 0x7f17600b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1765d30640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f176006c5f0 0x7f17600b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 -- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f176006c5f0 msgr2=0x7f17600b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f176006c5f0 0x7f17600b61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 -- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17600b25c0 msgr2=0x7f17600b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17600b25c0 0x7f17600b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 -- 172.21.15.106:0/1056621685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176009ee40 con 0x7f17600b4bd0 2024-04-25T11:20:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f176552f640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17600b25c0 0x7f17600b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1766531640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17600b4bd0 0x7f176014a3f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f175800c4a0 tx=0x7f175800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f175e7fc640 1 -- 172.21.15.106:0/1056621685 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758015070 con 0x7f17600b4bd0 2024-04-25T11:20:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f175e7fc640 1 -- 172.21.15.106:0/1056621685 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1758004240 con 0x7f17600b4bd0 2024-04-25T11:20:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f175e7fc640 1 -- 172.21.15.106:0/1056621685 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758003620 con 0x7f17600b4bd0 2024-04-25T11:20:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176009f120 con 0x7f17600b4bd0 2024-04-25T11:20:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176009f6e0 con 0x7f17600b4bd0 2024-04-25T11:20:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.620+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176009ebb0 con 0x7f17600b4bd0 2024-04-25T11:20:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.624+0000 7f175e7fc640 1 -- 172.21.15.106:0/1056621685 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17580037c0 con 0x7f17600b4bd0 2024-04-25T11:20:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.624+0000 7f175e7fc640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f174c06f400 0x7f174c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.624+0000 7f175e7fc640 1 -- 172.21.15.106:0/1056621685 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f175808f070 con 0x7f17600b4bd0 2024-04-25T11:20:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.624+0000 7f1765d30640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f174c06f400 0x7f174c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.624+0000 7f1765d30640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f174c06f400 0x7f174c071910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e90 tx=0x7f175000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.628+0000 7f175e7fc640 1 -- 172.21.15.106:0/1056621685 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f175805bec0 con 0x7f17600b4bd0 2024-04-25T11:20:03.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1760071630 con 0x7f17600b4bd0 2024-04-25T11:20:03.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f175e7fc640 1 -- 172.21.15.106:0/1056621685 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f175805b7c0 con 0x7f17600b4bd0 2024-04-25T11:20:03.775 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f174c06f400 msgr2=0x7f174c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f174c06f400 0x7f174c071910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e90 tx=0x7f175000b040 comp rx=0 tx=0).stop 2024-04-25T11:20:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17600b4bd0 msgr2=0x7f176014a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17600b4bd0 0x7f176014a3f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f175800c4a0 tx=0x7f175800c960 comp rx=0 tx=0).stop 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 shutdown_connections 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f174c06f400 0x7f174c071910 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17600b4bd0 0x7f176014a3f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17600b25c0 0x7f17600b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 --2- 172.21.15.106:0/1056621685 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f176006c5f0 0x7f17600b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 >> 172.21.15.106:0/1056621685 conn(0x7f17600b40b0 msgr2=0x7f176015f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 shutdown_connections 2024-04-25T11:20:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.776+0000 7f1767f94640 1 -- 172.21.15.106:0/1056621685 wait complete. 2024-04-25T11:20:03.797 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:03.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1aac21640 1 Processor -- start 2024-04-25T11:20:03.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1aac21640 1 -- start start 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1aac21640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc1a415fd10 0x7fc1a40b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1aac21640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc1a40d2150 0x7fc1a40d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1aac21640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a40d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1aac21640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a40d1350 con 0x7fc1a40d2150 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1aac21640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a40d2aa0 con 0x7fc1a415fd10 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1aac21640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a40b9270 con 0x7fc1a40d44a0 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc1a40d2150 0x7fc1a40d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc1a40d2150 0x7fc1a40d2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45780/0 (socket says 172.21.15.106:45780) 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a3fff640 1 -- 172.21.15.106:0/1810362098 learned_addr learned my addr 172.21.15.106:0/1810362098 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a89bd640 1 --2- 172.21.15.106:0/1810362098 >> v2:172.21.15.134:3300/0 conn(0x7fc1a415fd10 0x7fc1a40b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a91be640 1 --2- 172.21.15.106:0/1810362098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a40d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a3fff640 1 -- 172.21.15.106:0/1810362098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 msgr2=0x7fc1a40d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a3fff640 1 --2- 172.21.15.106:0/1810362098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a40d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a3fff640 1 -- 172.21.15.106:0/1810362098 >> v2:172.21.15.134:3300/0 conn(0x7fc1a415fd10 msgr2=0x7fc1a40b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a3fff640 1 --2- 172.21.15.106:0/1810362098 >> v2:172.21.15.134:3300/0 conn(0x7fc1a415fd10 0x7fc1a40b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.952+0000 7fc1a3fff640 1 -- 172.21.15.106:0/1810362098 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a4163cc0 con 0x7fc1a40d2150 2024-04-25T11:20:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1a3fff640 1 --2- 172.21.15.106:0/1810362098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a40d2560 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc1a408e730 tx=0x7fc19401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b773679b156ed90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1a37fe640 1 -- 172.21.15.106:0/1810362098 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1940b0070 con 0x7fc1a40d2150 2024-04-25T11:20:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1a37fe640 1 -- 172.21.15.106:0/1810362098 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1940ab600 con 0x7fc1a40d2150 2024-04-25T11:20:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1a37fe640 1 -- 172.21.15.106:0/1810362098 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1940ab9d0 con 0x7fc1a40d2150 2024-04-25T11:20:03.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 -- 172.21.15.106:0/1810362098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 msgr2=0x7fc1a40d2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 --2- 172.21.15.106:0/1810362098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a40d2560 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc1a408e730 tx=0x7fc19401c520 comp rx=0 tx=0).stop 2024-04-25T11:20:03.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 -- 172.21.15.106:0/1810362098 shutdown_connections 2024-04-25T11:20:03.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 --2- 172.21.15.106:0/1810362098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a40d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 --2- 172.21.15.106:0/1810362098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a40d2560 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 --2- 172.21.15.106:0/1810362098 >> v2:172.21.15.134:3300/0 conn(0x7fc1a415fd10 0x7fc1a40b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 -- 172.21.15.106:0/1810362098 >> 172.21.15.106:0/1810362098 conn(0x7fc1a406d7f0 msgr2=0x7fc1a406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 -- 172.21.15.106:0/1810362098 shutdown_connections 2024-04-25T11:20:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.956+0000 7fc1aac21640 1 -- 172.21.15.106:0/1810362098 wait complete. 2024-04-25T11:20:03.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 Processor -- start 2024-04-25T11:20:03.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 -- start start 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a4154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a4159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a415fd10 0x7fc1a4159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41638e0 con 0x7fc1a40d2150 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4160120 con 0x7fc1a415fd10 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a4163a60 con 0x7fc1a40d44a0 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a4154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a4154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45786/0 (socket says 172.21.15.106:45786) 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 -- 172.21.15.106:0/2275750848 learned_addr learned my addr 172.21.15.106:0/2275750848 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a91be640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a415fd10 0x7fc1a4159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a3fff640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a4159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 -- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 msgr2=0x7fc1a4159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a4159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 -- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a415fd10 msgr2=0x7fc1a4159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a415fd10 0x7fc1a4159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 -- 172.21.15.106:0/2275750848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a4162d00 con 0x7fc1a40d2150 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a3fff640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a4159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a89bd640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a4154110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc18c007e40 tx=0x7fc18c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a17fa640 1 -- 172.21.15.106:0/2275750848 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c014620 con 0x7fc1a40d2150 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a17fa640 1 -- 172.21.15.106:0/2275750848 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c019070 con 0x7fc1a40d2150 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a416a950 con 0x7fc1a40d2150 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1a17fa640 1 -- 172.21.15.106:0/2275750848 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c014a00 con 0x7fc1a40d2150 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a4158e40 con 0x7fc1a40d2150 2024-04-25T11:20:03.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.960+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a4050750 con 0x7fc1a40d2150 2024-04-25T11:20:03.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.964+0000 7fc1a17fa640 1 -- 172.21.15.106:0/2275750848 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc18c014ba0 con 0x7fc1a40d2150 2024-04-25T11:20:03.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.964+0000 7fc1a17fa640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc18406f430 0x7fc184071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.964+0000 7fc1a17fa640 1 -- 172.21.15.106:0/2275750848 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc18c090ad0 con 0x7fc1a40d2150 2024-04-25T11:20:03.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.964+0000 7fc1a3fff640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc18406f430 0x7fc184071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.964+0000 7fc1a3fff640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc18406f430 0x7fc184071940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc1940ad000 tx=0x7fc194008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:03.968+0000 7fc1a17fa640 1 -- 172.21.15.106:0/2275750848 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc18c05d350 con 0x7fc1a40d2150 2024-04-25T11:20:04.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.128+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1a40d3be0 con 0x7fc1a40d2150 2024-04-25T11:20:04.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.128+0000 7fc1a17fa640 1 -- 172.21.15.106:0/2275750848 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc18c061650 con 0x7fc1a40d2150 2024-04-25T11:20:04.128 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc18406f430 msgr2=0x7fc184071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc18406f430 0x7fc184071940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc1940ad000 tx=0x7fc194008040 comp rx=0 tx=0).stop 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 msgr2=0x7fc1a4154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a4154110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc18c007e40 tx=0x7fc18c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 shutdown_connections 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a415fd10 0x7fc1a4159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a40d44a0 0x7fc1a4159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc18406f430 0x7fc184071940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 --2- 172.21.15.106:0/2275750848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a40d2150 0x7fc1a4154110 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 >> 172.21.15.106:0/2275750848 conn(0x7fc1a406d7f0 msgr2=0x7fc1a406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:04.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 shutdown_connections 2024-04-25T11:20:04.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.132+0000 7fc1aac21640 1 -- 172.21.15.106:0/2275750848 wait complete. 2024-04-25T11:20:04.151 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:04.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.300+0000 7fd41c892640 1 Processor -- start 2024-04-25T11:20:04.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.300+0000 7fd41c892640 1 -- start start 2024-04-25T11:20:04.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd41c892640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd418160fc0 0x7fd4180b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd41c892640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd4180b5110 0x7fd41806c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd41c892640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4180b5520 con 0x7fd41815cae0 2024-04-25T11:20:04.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd41c892640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd4180ce2a0 con 0x7fd418160fc0 2024-04-25T11:20:04.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd41c892640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd4180b4460 con 0x7fd4180b5110 2024-04-25T11:20:04.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd415d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd418160fc0 0x7fd4180b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd415d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd418160fc0 0x7fd4180b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45794/0 (socket says 172.21.15.106:45794) 2024-04-25T11:20:04.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd415d9b640 1 -- 172.21.15.106:0/2753643359 learned_addr learned my addr 172.21.15.106:0/2753643359 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:04.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd416d9d640 1 --2- 172.21.15.106:0/2753643359 >> v2:172.21.15.134:3300/0 conn(0x7fd4180b5110 0x7fd41806c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd415d9b640 1 -- 172.21.15.106:0/2753643359 >> v2:172.21.15.134:3300/0 conn(0x7fd4180b5110 msgr2=0x7fd41806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:04.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd415d9b640 1 --2- 172.21.15.106:0/2753643359 >> v2:172.21.15.134:3300/0 conn(0x7fd4180b5110 0x7fd41806c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd415d9b640 1 -- 172.21.15.106:0/2753643359 >> v1:172.21.15.134:6789/0 conn(0x7fd41815cae0 legacy=0x7fd4180cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:04.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd415d9b640 1 -- 172.21.15.106:0/2753643359 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4181613d0 con 0x7fd418160fc0 2024-04-25T11:20:04.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd416d9d640 1 --2- 172.21.15.106:0/2753643359 >> v2:172.21.15.134:3300/0 conn(0x7fd4180b5110 0x7fd41806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:04.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd415d9b640 1 --2- 172.21.15.106:0/2753643359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd4180b4bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd4181616b0 tx=0x7fd40c01c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22c21e8961f9140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:04.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd41559a640 1 -- 172.21.15.106:0/2753643359 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0b0070 con 0x7fd418160fc0 2024-04-25T11:20:04.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd41559a640 1 -- 172.21.15.106:0/2753643359 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40c0ab600 con 0x7fd418160fc0 2024-04-25T11:20:04.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.304+0000 7fd41559a640 1 -- 172.21.15.106:0/2753643359 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0ab9d0 con 0x7fd418160fc0 2024-04-25T11:20:04.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- 172.21.15.106:0/2753643359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 msgr2=0x7fd4180b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 --2- 172.21.15.106:0/2753643359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd4180b4bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd4181616b0 tx=0x7fd40c01c270 comp rx=0 tx=0).stop 2024-04-25T11:20:04.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- 172.21.15.106:0/2753643359 shutdown_connections 2024-04-25T11:20:04.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 --2- 172.21.15.106:0/2753643359 >> v2:172.21.15.134:3300/0 conn(0x7fd4180b5110 0x7fd41806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 --2- 172.21.15.106:0/2753643359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd4180b4bd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- 172.21.15.106:0/2753643359 >> 172.21.15.106:0/2753643359 conn(0x7fd4180b3560 msgr2=0x7fd4180b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:04.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- 172.21.15.106:0/2753643359 shutdown_connections 2024-04-25T11:20:04.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- 172.21.15.106:0/2753643359 wait complete. 2024-04-25T11:20:04.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 Processor -- start 2024-04-25T11:20:04.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- start start 2024-04-25T11:20:04.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4180b5110 0x7fd418147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd41815cae0 0x7fd4181567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd418155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd41806d170 con 0x7fd418160fc0 2024-04-25T11:20:04.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4180b4740 con 0x7fd4180b5110 2024-04-25T11:20:04.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd41c892640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd4180cfde0 con 0x7fd41815cae0 2024-04-25T11:20:04.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd416d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd418155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd416d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd418155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45798/0 (socket says 172.21.15.106:45798) 2024-04-25T11:20:04.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.308+0000 7fd416d9d640 1 -- 172.21.15.106:0/377420275 learned_addr learned my addr 172.21.15.106:0/377420275 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:04.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd41659c640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4180b5110 0x7fd418147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd415d9b640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd41815cae0 0x7fd4181567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd416d9d640 1 -- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd41815cae0 msgr2=0x7fd4181567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd416d9d640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd41815cae0 0x7fd4181567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd416d9d640 1 -- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4180b5110 msgr2=0x7fd418147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd416d9d640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4180b5110 0x7fd418147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd416d9d640 1 -- 172.21.15.106:0/377420275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41806f7e0 con 0x7fd418160fc0 2024-04-25T11:20:04.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd41659c640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4180b5110 0x7fd418147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:04.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd416d9d640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd418155c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd41000cd80 tx=0x7fd41000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:04.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd406ffd640 1 -- 172.21.15.106:0/377420275 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41000f680 con 0x7fd418160fc0 2024-04-25T11:20:04.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd406ffd640 1 -- 172.21.15.106:0/377420275 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41000f840 con 0x7fd418160fc0 2024-04-25T11:20:04.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41814f5d0 con 0x7fd418160fc0 2024-04-25T11:20:04.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd406ffd640 1 -- 172.21.15.106:0/377420275 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41000fc30 con 0x7fd418160fc0 2024-04-25T11:20:04.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41814fb90 con 0x7fd418160fc0 2024-04-25T11:20:04.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd418050750 con 0x7fd418160fc0 2024-04-25T11:20:04.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd406ffd640 1 -- 172.21.15.106:0/377420275 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4100213f0 con 0x7fd418160fc0 2024-04-25T11:20:04.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd406ffd640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd3f806f430 0x7fd3f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd406ffd640 1 -- 172.21.15.106:0/377420275 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd410090020 con 0x7fd418160fc0 2024-04-25T11:20:04.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd41659c640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd3f806f430 0x7fd3f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.312+0000 7fd41659c640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd3f806f430 0x7fd3f8071940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd40800a850 tx=0x7fd408008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:04.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.316+0000 7fd406ffd640 1 -- 172.21.15.106:0/377420275 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd410014030 con 0x7fd418160fc0 2024-04-25T11:20:04.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.476+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd4180b0db0 con 0x7fd418160fc0 2024-04-25T11:20:04.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.476+0000 7fd406ffd640 1 -- 172.21.15.106:0/377420275 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fd41005cf10 con 0x7fd418160fc0 2024-04-25T11:20:04.476 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd3f806f430 msgr2=0x7fd3f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd3f806f430 0x7fd3f8071940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd40800a850 tx=0x7fd408008040 comp rx=0 tx=0).stop 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 msgr2=0x7fd418155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd418155c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd41000cd80 tx=0x7fd41000a650 comp rx=0 tx=0).stop 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 shutdown_connections 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd3f806f430 0x7fd3f8071940 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd418160fc0 0x7fd418155c30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd41815cae0 0x7fd4181567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 --2- 172.21.15.106:0/377420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4180b5110 0x7fd418147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 >> 172.21.15.106:0/377420275 conn(0x7fd4180b3560 msgr2=0x7fd4180cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:04.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 shutdown_connections 2024-04-25T11:20:04.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.480+0000 7fd41c892640 1 -- 172.21.15.106:0/377420275 wait complete. 2024-04-25T11:20:04.496 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:04.497 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:04.502 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:04.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe711541640 1 Processor -- start 2024-04-25T11:20:04.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe711541640 1 -- start start 2024-04-25T11:20:04.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe711541640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe70c156f90 0x7fe70c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe711541640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe70c1573a0 con 0x7fe70c0b5a80 2024-04-25T11:20:04.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe711541640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe70c15d220 con 0x7fe70c06fea0 2024-04-25T11:20:04.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe711541640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe70c0d1f90 con 0x7fe70c156f90 2024-04-25T11:20:04.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe70affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe70c156f90 0x7fe70c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe70b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe70c0b5a80 0x7fe70c0b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42124/0 (socket says 172.21.15.106:42124) 2024-04-25T11:20:04.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe70b7fe640 1 -- 172.21.15.106:0/3261010459 learned_addr learned my addr 172.21.15.106:0/3261010459 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:04.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe70affd640 1 -- 172.21.15.106:0/3261010459 >> v1:172.21.15.134:6789/0 conn(0x7fe70c06fea0 legacy=0x7fe70c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe70affd640 1 -- 172.21.15.106:0/3261010459 >> v1:172.21.15.106:6789/0 conn(0x7fe70c0b5a80 legacy=0x7fe70c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe709ffb640 1 -- 172.21.15.106:0/3261010459 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2881321555 0 0) 0x7fe70c1573a0 con 0x7fe70c0b5a80 2024-04-25T11:20:04.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.700+0000 7fe70affd640 1 -- 172.21.15.106:0/3261010459 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c15d500 con 0x7fe70c156f90 2024-04-25T11:20:04.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe70affd640 1 --2- 172.21.15.106:0/3261010459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c156f90 0x7fe70c0d1a50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009d90 tx=0x7fe6fc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c53fe712b7c0005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:04.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe709ffb640 1 -- 172.21.15.106:0/3261010459 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0b2070 con 0x7fe70c156f90 2024-04-25T11:20:04.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe709ffb640 1 -- 172.21.15.106:0/3261010459 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc0a5890 con 0x7fe70c156f90 2024-04-25T11:20:04.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe709ffb640 1 -- 172.21.15.106:0/3261010459 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0a5c60 con 0x7fe70c156f90 2024-04-25T11:20:04.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 -- 172.21.15.106:0/3261010459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c156f90 msgr2=0x7fe70c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 --2- 172.21.15.106:0/3261010459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c156f90 0x7fe70c0d1a50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009d90 tx=0x7fe6fc01c8e0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 -- 172.21.15.106:0/3261010459 shutdown_connections 2024-04-25T11:20:04.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 --2- 172.21.15.106:0/3261010459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c156f90 0x7fe70c0d1a50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 -- 172.21.15.106:0/3261010459 >> 172.21.15.106:0/3261010459 conn(0x7fe70c06c7c0 msgr2=0x7fe70c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:04.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 -- 172.21.15.106:0/3261010459 shutdown_connections 2024-04-25T11:20:04.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 -- 172.21.15.106:0/3261010459 wait complete. 2024-04-25T11:20:04.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 Processor -- start 2024-04-25T11:20:04.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.704+0000 7fe711541640 1 -- start start 2024-04-25T11:20:04.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c06fea0 0x7fe70c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe70c0b5a80 0x7fe70c15f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe70c156f90 0x7fe70c15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c0b65a0 con 0x7fe70c06fea0 2024-04-25T11:20:04.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c15dda0 con 0x7fe70c0b5a80 2024-04-25T11:20:04.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe70c0b6720 con 0x7fe70c156f90 2024-04-25T11:20:04.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c06fea0 0x7fe70c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c06fea0 0x7fe70c1615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45822/0 (socket says 172.21.15.106:45822) 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 -- 172.21.15.106:0/2716203754 learned_addr learned my addr 172.21.15.106:0/2716203754 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70a7fc640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe70c0b5a80 0x7fe70c15f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70b7fe640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe70c156f90 0x7fe70c15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 -- 172.21.15.106:0/2716203754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe70c156f90 msgr2=0x7fe70c15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe70c156f90 0x7fe70c15b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 -- 172.21.15.106:0/2716203754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe70c0b5a80 msgr2=0x7fe70c15f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe70c0b5a80 0x7fe70c15f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 -- 172.21.15.106:0/2716203754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c15e730 con 0x7fe70c06fea0 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe70affd640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c06fea0 0x7fe70c1615f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0194f0 tx=0x7fe6fc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe7037fe640 1 -- 172.21.15.106:0/2716203754 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0b2040 con 0x7fe70c06fea0 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe7037fe640 1 -- 172.21.15.106:0/2716203754 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc0bf070 con 0x7fe70c06fea0 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe7037fe640 1 -- 172.21.15.106:0/2716203754 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0b8890 con 0x7fe70c06fea0 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c15ea10 con 0x7fe70c06fea0 2024-04-25T11:20:04.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c15efd0 con 0x7fe70c06fea0 2024-04-25T11:20:04.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.708+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c0759a0 con 0x7fe70c06fea0 2024-04-25T11:20:04.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.712+0000 7fe7037fe640 1 -- 172.21.15.106:0/2716203754 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe6fc0b8a30 con 0x7fe70c06fea0 2024-04-25T11:20:04.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.712+0000 7fe7037fe640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe6ec06f430 0x7fe6ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:04.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.712+0000 7fe7037fe640 1 -- 172.21.15.106:0/2716203754 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe6fc129e60 con 0x7fe70c06fea0 2024-04-25T11:20:04.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.712+0000 7fe70a7fc640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe6ec06f430 0x7fe6ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:04.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.712+0000 7fe70a7fc640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe6ec06f430 0x7fe6ec071940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe70c069b80 tx=0x7fe6f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:04.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.716+0000 7fe7037fe640 1 -- 172.21.15.106:0/2716203754 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe6fc0c8020 con 0x7fe70c06fea0 2024-04-25T11:20:04.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.864+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe70c0008e0 con 0x7fe70c06fea0 2024-04-25T11:20:04.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.868+0000 7fe7037fe640 1 -- 172.21.15.106:0/2716203754 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7fe6fc016560 con 0x7fe70c06fea0 2024-04-25T11:20:04.867 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:04.867 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:19:28.407152+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:04.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe6ec06f430 msgr2=0x7fe6ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe6ec06f430 0x7fe6ec071940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe70c069b80 tx=0x7fe6f4009040 comp rx=0 tx=0).stop 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c06fea0 msgr2=0x7fe70c1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c06fea0 0x7fe70c1615f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0194f0 tx=0x7fe6fc019680 comp rx=0 tx=0).stop 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 shutdown_connections 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe70c156f90 0x7fe70c15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe70c0b5a80 0x7fe70c15f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe6ec06f430 0x7fe6ec071940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 --2- 172.21.15.106:0/2716203754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe70c06fea0 0x7fe70c1615f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 >> 172.21.15.106:0/2716203754 conn(0x7fe70c06c7c0 msgr2=0x7fe70c0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 shutdown_connections 2024-04-25T11:20:04.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:04.872+0000 7fe711541640 1 -- 172.21.15.106:0/2716203754 wait complete. 2024-04-25T11:20:04.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:04.890 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:05.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.028+0000 7fa5f3553640 1 Processor -- start 2024-04-25T11:20:05.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.028+0000 7fa5f3553640 1 -- start start 2024-04-25T11:20:05.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.028+0000 7fa5f3553640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa5ec143fa0 0x7fa5ec0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.028+0000 7fa5f3553640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ec073300 con 0x7fa5ec072a20 2024-04-25T11:20:05.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.028+0000 7fa5f3553640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ec0735e0 con 0x7fa5ec0d1e50 2024-04-25T11:20:05.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.028+0000 7fa5f3553640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa5ec0738c0 con 0x7fa5ec143fa0 2024-04-25T11:20:05.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.028+0000 7fa5f0aee640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa5ec072a20 0x7fa5ec0d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42130/0 (socket says 172.21.15.106:42130) 2024-04-25T11:20:05.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.028+0000 7fa5f0aee640 1 -- 172.21.15.106:0/2787313401 learned_addr learned my addr 172.21.15.106:0/2787313401 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:05.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f12ef640 1 --2- 172.21.15.106:0/2787313401 >> v2:172.21.15.134:3300/0 conn(0x7fa5ec143fa0 0x7fa5ec0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821311430 0 0) 0x7fa5ec073300 con 0x7fa5ec072a20 2024-04-25T11:20:05.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d8003530 con 0x7fa5ec072a20 2024-04-25T11:20:05.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1191801463 0 0) 0x7fa5d8003530 con 0x7fa5ec072a20 2024-04-25T11:20:05.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 >> v2:172.21.15.134:3300/0 conn(0x7fa5ec143fa0 msgr2=0x7fa5ec0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:05.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 --2- 172.21.15.106:0/2787313401 >> v2:172.21.15.134:3300/0 conn(0x7fa5ec143fa0 0x7fa5ec0cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 >> v1:172.21.15.134:6789/0 conn(0x7fa5ec0d1e50 legacy=0x7fa5ec0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec0b5040 con 0x7fa5ec072a20 2024-04-25T11:20:05.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fa5dc001680 con 0x7fa5ec072a20 2024-04-25T11:20:05.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 >> v1:172.21.15.106:6789/0 conn(0x7fa5ec072a20 legacy=0x7fa5ec0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d80039c0 0x7fa5d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5d8004810 0x7fa5d8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d80051d0 0x7fa5d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec0738c0 con 0x7fa5d8004810 2024-04-25T11:20:05.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d800c170 con 0x7fa5d80051d0 2024-04-25T11:20:05.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d800c370 con 0x7fa5d80039c0 2024-04-25T11:20:05.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f0aee640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5d8004810 0x7fa5d8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f0aee640 1 -- 172.21.15.106:0/2787313401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d80039c0 msgr2=0x7fa5d8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:05.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f0aee640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d80039c0 0x7fa5d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f0aee640 1 -- 172.21.15.106:0/2787313401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d80051d0 msgr2=0x7fa5d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:05.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f0aee640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d80051d0 0x7fa5d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f1af0640 1 -- 172.21.15.106:0/2787313401 reap_dead start 2024-04-25T11:20:05.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f0aee640 1 -- 172.21.15.106:0/2787313401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d8003530 con 0x7fa5d8004810 2024-04-25T11:20:05.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5f0aee640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5d8004810 0x7fa5d8004c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00c1c0 tx=0x7fa5dc01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acac8a8834c9757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc007d00 con 0x7fa5d8004810 2024-04-25T11:20:05.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc005b00 con 0x7fa5d8004810 2024-04-25T11:20:05.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.032+0000 7fa5e3fff640 1 -- 172.21.15.106:0/2787313401 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc007070 con 0x7fa5d8004810 2024-04-25T11:20:05.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- 172.21.15.106:0/2787313401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5d8004810 msgr2=0x7fa5d8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5d8004810 0x7fa5d8004c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00c1c0 tx=0x7fa5dc01f3d0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- 172.21.15.106:0/2787313401 shutdown_connections 2024-04-25T11:20:05.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 --2- 172.21.15.106:0/2787313401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5d8004810 0x7fa5d8004c20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- 172.21.15.106:0/2787313401 >> 172.21.15.106:0/2787313401 conn(0x7fa5ec06c9f0 msgr2=0x7fa5ec06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:05.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- 172.21.15.106:0/2787313401 shutdown_connections 2024-04-25T11:20:05.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- 172.21.15.106:0/2787313401 wait complete. 2024-04-25T11:20:05.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 Processor -- start 2024-04-25T11:20:05.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- start start 2024-04-25T11:20:05.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d8004810 0x7fa5ec13f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5ec0ac390 0x7fa5ec0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5ec1406b0 0x7fa5ec140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec072fb0 con 0x7fa5ec1406b0 2024-04-25T11:20:05.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec072e30 con 0x7fa5d8004810 2024-04-25T11:20:05.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f3553640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ec0cd930 con 0x7fa5ec0ac390 2024-04-25T11:20:05.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f1af0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5ec1406b0 0x7fa5ec140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f1af0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5ec1406b0 0x7fa5ec140ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45836/0 (socket says 172.21.15.106:45836) 2024-04-25T11:20:05.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.036+0000 7fa5f1af0640 1 -- 172.21.15.106:0/1834780217 learned_addr learned my addr 172.21.15.106:0/1834780217 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:05.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f12ef640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d8004810 0x7fa5ec13f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f0aee640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5ec0ac390 0x7fa5ec0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f1af0640 1 -- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5ec0ac390 msgr2=0x7fa5ec0ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f1af0640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5ec0ac390 0x7fa5ec0ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f1af0640 1 -- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d8004810 msgr2=0x7fa5ec13f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f1af0640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d8004810 0x7fa5ec13f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f1af0640 1 -- 172.21.15.106:0/1834780217 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec141000 con 0x7fa5ec1406b0 2024-04-25T11:20:05.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f0aee640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5ec0ac390 0x7fa5ec0ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:05.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f12ef640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d8004810 0x7fa5ec13f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:05.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f1af0640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5ec1406b0 0x7fa5ec140ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0d0100 tx=0x7fa5e400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5e1ffb640 1 -- 172.21.15.106:0/1834780217 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4017070 con 0x7fa5ec1406b0 2024-04-25T11:20:05.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5e1ffb640 1 -- 172.21.15.106:0/1834780217 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e4005d50 con 0x7fa5ec1406b0 2024-04-25T11:20:05.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5e1ffb640 1 -- 172.21.15.106:0/1834780217 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400a850 con 0x7fa5ec1406b0 2024-04-25T11:20:05.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec13f610 con 0x7fa5ec1406b0 2024-04-25T11:20:05.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec13fbd0 con 0x7fa5ec1406b0 2024-04-25T11:20:05.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ec0acb50 con 0x7fa5ec1406b0 2024-04-25T11:20:05.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5e1ffb640 1 -- 172.21.15.106:0/1834780217 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa5e401a070 con 0x7fa5ec1406b0 2024-04-25T11:20:05.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5e1ffb640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa5d006f400 0x7fa5d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5e1ffb640 1 -- 172.21.15.106:0/1834780217 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa5e408e480 con 0x7fa5ec1406b0 2024-04-25T11:20:05.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f12ef640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa5d006f400 0x7fa5d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.040+0000 7fa5f12ef640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa5d006f400 0x7fa5d0071910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80047c0 tx=0x7fa5e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.044+0000 7fa5e1ffb640 1 -- 172.21.15.106:0/1834780217 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5e405b400 con 0x7fa5ec1406b0 2024-04-25T11:20:05.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.192+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5ec0b02d0 con 0x7fa5ec1406b0 2024-04-25T11:20:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5e1ffb640 1 -- 172.21.15.106:0/1834780217 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa5e40633f0 con 0x7fa5ec1406b0 2024-04-25T11:20:05.194 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa5d006f400 msgr2=0x7fa5d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa5d006f400 0x7fa5d0071910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80047c0 tx=0x7fa5e8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5ec1406b0 msgr2=0x7fa5ec140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5ec1406b0 0x7fa5ec140ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0d0100 tx=0x7fa5e400cb10 comp rx=0 tx=0).stop 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 shutdown_connections 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa5d006f400 0x7fa5d0071910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5ec1406b0 0x7fa5ec140ac0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5ec0ac390 0x7fa5ec0ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 --2- 172.21.15.106:0/1834780217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d8004810 0x7fa5ec13f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 >> 172.21.15.106:0/1834780217 conn(0x7fa5ec06c9f0 msgr2=0x7fa5ec143630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:05.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 shutdown_connections 2024-04-25T11:20:05.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.196+0000 7fa5f3553640 1 -- 172.21.15.106:0/1834780217 wait complete. 2024-04-25T11:20:05.214 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:05.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb4738640 1 Processor -- start 2024-04-25T11:20:05.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb4738640 1 -- start start 2024-04-25T11:20:05.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb4738640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fddac15fd10 0x7fddac0b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb4738640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fddac0d2150 0x7fddac0d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb4738640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddac0d1350 con 0x7fddac0d44a0 2024-04-25T11:20:05.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb4738640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fddac0d2aa0 con 0x7fddac15fd10 2024-04-25T11:20:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb4738640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fddac0b9270 con 0x7fddac0d2150 2024-04-25T11:20:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb2cd5640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fddac0d44a0 0x7fddac0d48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42144/0 (socket says 172.21.15.106:42144) 2024-04-25T11:20:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb2cd5640 1 -- 172.21.15.106:0/2966345888 learned_addr learned my addr 172.21.15.106:0/2966345888 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb24d4640 1 --2- 172.21.15.106:0/2966345888 >> v2:172.21.15.106:3300/0 conn(0x7fddac15fd10 0x7fddac0b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb1cd3640 1 --2- 172.21.15.106:0/2966345888 >> v2:172.21.15.134:3300/0 conn(0x7fddac0d2150 0x7fddac0d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb14d2640 1 -- 172.21.15.106:0/2966345888 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3167933143 0 0) 0x7fddac0d1350 con 0x7fddac0d44a0 2024-04-25T11:20:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb14d2640 1 -- 172.21.15.106:0/2966345888 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd98003530 con 0x7fddac0d44a0 2024-04-25T11:20:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb24d4640 1 -- 172.21.15.106:0/2966345888 >> v2:172.21.15.134:3300/0 conn(0x7fddac0d2150 msgr2=0x7fddac0d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb24d4640 1 --2- 172.21.15.106:0/2966345888 >> v2:172.21.15.134:3300/0 conn(0x7fddac0d2150 0x7fddac0d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb24d4640 1 -- 172.21.15.106:0/2966345888 >> v1:172.21.15.106:6789/0 conn(0x7fddac0d44a0 legacy=0x7fddac0d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.368+0000 7fddb24d4640 1 -- 172.21.15.106:0/2966345888 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac163cc0 con 0x7fddac15fd10 2024-04-25T11:20:05.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.372+0000 7fddb24d4640 1 --2- 172.21.15.106:0/2966345888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac0b8d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddac08ea50 tx=0x7fdda801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cbbce2d88e223f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.372+0000 7fddb14d2640 1 -- 172.21.15.106:0/2966345888 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda80b0070 con 0x7fddac15fd10 2024-04-25T11:20:05.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.372+0000 7fddb14d2640 1 -- 172.21.15.106:0/2966345888 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda80ab600 con 0x7fddac15fd10 2024-04-25T11:20:05.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.372+0000 7fddb14d2640 1 -- 172.21.15.106:0/2966345888 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda80ab9b0 con 0x7fddac15fd10 2024-04-25T11:20:05.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.372+0000 7fddb4738640 1 -- 172.21.15.106:0/2966345888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 msgr2=0x7fddac0b8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.372+0000 7fddb4738640 1 --2- 172.21.15.106:0/2966345888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac0b8d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddac08ea50 tx=0x7fdda801c480 comp rx=0 tx=0).stop 2024-04-25T11:20:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.372+0000 7fddb4738640 1 -- 172.21.15.106:0/2966345888 shutdown_connections 2024-04-25T11:20:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 --2- 172.21.15.106:0/2966345888 >> v2:172.21.15.134:3300/0 conn(0x7fddac0d2150 0x7fddac0d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 --2- 172.21.15.106:0/2966345888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac0b8d30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 -- 172.21.15.106:0/2966345888 >> 172.21.15.106:0/2966345888 conn(0x7fddac06d7f0 msgr2=0x7fddac06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 -- 172.21.15.106:0/2966345888 shutdown_connections 2024-04-25T11:20:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 -- 172.21.15.106:0/2966345888 wait complete. 2024-04-25T11:20:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 Processor -- start 2024-04-25T11:20:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 -- start start 2024-04-25T11:20:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac0d2150 0x7fddac14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac0d44a0 0x7fddac153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fddac1638e0 con 0x7fddac15fd10 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fddac160120 con 0x7fddac0d2150 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb4738640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fddac163a60 con 0x7fddac0d44a0 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb2cd5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb2cd5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45860/0 (socket says 172.21.15.106:45860) 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb2cd5640 1 -- 172.21.15.106:0/380826129 learned_addr learned my addr 172.21.15.106:0/380826129 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb24d4640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac0d2150 0x7fddac14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb1cd3640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac0d44a0 0x7fddac153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb2cd5640 1 -- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac0d44a0 msgr2=0x7fddac153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb2cd5640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac0d44a0 0x7fddac153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb2cd5640 1 -- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac0d2150 msgr2=0x7fddac14c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb2cd5640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac0d2150 0x7fddac14c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb2cd5640 1 -- 172.21.15.106:0/380826129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac15b0e0 con 0x7fddac15fd10 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.376+0000 7fddb1cd3640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac0d44a0 0x7fddac153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fddb2cd5640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac15dcb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdda400cad0 tx=0x7fdda400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fdda2ffd640 1 -- 172.21.15.106:0/380826129 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4017070 con 0x7fddac15fd10 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fdda2ffd640 1 -- 172.21.15.106:0/380826129 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda400abd0 con 0x7fddac15fd10 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddac1546d0 con 0x7fddac15fd10 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fdda2ffd640 1 -- 172.21.15.106:0/380826129 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4012400 con 0x7fddac15fd10 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddac154c90 con 0x7fddac15fd10 2024-04-25T11:20:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddac050750 con 0x7fddac15fd10 2024-04-25T11:20:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fdda2ffd640 1 -- 172.21.15.106:0/380826129 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdda401a070 con 0x7fddac15fd10 2024-04-25T11:20:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fdda2ffd640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdd9006f430 0x7fdd90071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fddb24d4640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdd9006f430 0x7fdd90071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fdda2ffd640 1 -- 172.21.15.106:0/380826129 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdda408f360 con 0x7fddac15fd10 2024-04-25T11:20:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.380+0000 7fddb24d4640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdd9006f430 0x7fdd90071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdda80ad000 tx=0x7fdda80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.384+0000 7fdda2ffd640 1 -- 172.21.15.106:0/380826129 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdda4021020 con 0x7fddac15fd10 2024-04-25T11:20:05.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fddac154420 con 0x7fddac15fd10 2024-04-25T11:20:05.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fdda2ffd640 1 -- 172.21.15.106:0/380826129 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fdda405c330 con 0x7fddac15fd10 2024-04-25T11:20:05.539 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:05.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdd9006f430 msgr2=0x7fdd90071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdd9006f430 0x7fdd90071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdda80ad000 tx=0x7fdda80a6000 comp rx=0 tx=0).stop 2024-04-25T11:20:05.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 msgr2=0x7fddac15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac15dcb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdda400cad0 tx=0x7fdda400cf90 comp rx=0 tx=0).stop 2024-04-25T11:20:05.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 shutdown_connections 2024-04-25T11:20:05.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdd9006f430 0x7fdd90071940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fddac15fd10 0x7fddac15dcb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac0d44a0 0x7fddac153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 --2- 172.21.15.106:0/380826129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac0d2150 0x7fddac14c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 >> 172.21.15.106:0/380826129 conn(0x7fddac06d7f0 msgr2=0x7fddac0d5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:05.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 shutdown_connections 2024-04-25T11:20:05.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.540+0000 7fddb4738640 1 -- 172.21.15.106:0/380826129 wait complete. 2024-04-25T11:20:05.559 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.704+0000 7fa5a2cf4640 1 Processor -- start 2024-04-25T11:20:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.704+0000 7fa5a2cf4640 1 -- start start 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a2cf4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa59c143fa0 0x7fa59c0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a2cf4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa59c072a20 0x7fa59c0d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a2cf4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59c073300 con 0x7fa59c072e30 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a2cf4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c0735e0 con 0x7fa59c143fa0 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a2cf4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c0738c0 con 0x7fa59c072a20 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a0a90640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa59c143fa0 0x7fa59c0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a1291640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa59c072e30 0x7fa59c0b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37198/0 (socket says 172.21.15.106:37198) 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a1291640 1 -- 172.21.15.106:0/3384961027 learned_addr learned my addr 172.21.15.106:0/3384961027 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa59bfff640 1 --2- 172.21.15.106:0/3384961027 >> v2:172.21.15.134:3300/0 conn(0x7fa59c072a20 0x7fa59c0d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a0a90640 1 -- 172.21.15.106:0/3384961027 >> v2:172.21.15.134:3300/0 conn(0x7fa59c072a20 msgr2=0x7fa59c0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a0a90640 1 --2- 172.21.15.106:0/3384961027 >> v2:172.21.15.134:3300/0 conn(0x7fa59c072a20 0x7fa59c0d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a0a90640 1 -- 172.21.15.106:0/3384961027 >> v1:172.21.15.106:6789/0 conn(0x7fa59c072e30 legacy=0x7fa59c0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.708+0000 7fa5a0a90640 1 -- 172.21.15.106:0/3384961027 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c0b5040 con 0x7fa59c143fa0 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a0a90640 1 --2- 172.21.15.106:0/3384961027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c0cd930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0721f0 tx=0x7fa59001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6961df7e0ba9c8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa59b7fe640 1 -- 172.21.15.106:0/3384961027 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900b0070 con 0x7fa59c143fa0 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa59b7fe640 1 -- 172.21.15.106:0/3384961027 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5900ab600 con 0x7fa59c143fa0 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa59b7fe640 1 -- 172.21.15.106:0/3384961027 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900ab9d0 con 0x7fa59c143fa0 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/3384961027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 msgr2=0x7fa59c0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/3384961027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c0cd930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0721f0 tx=0x7fa59001c450 comp rx=0 tx=0).stop 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/3384961027 shutdown_connections 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/3384961027 >> v2:172.21.15.134:3300/0 conn(0x7fa59c072a20 0x7fa59c0d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/3384961027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c0cd930 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/3384961027 >> 172.21.15.106:0/3384961027 conn(0x7fa59c06c9f0 msgr2=0x7fa59c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/3384961027 shutdown_connections 2024-04-25T11:20:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/3384961027 wait complete. 2024-04-25T11:20:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 Processor -- start 2024-04-25T11:20:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.712+0000 7fa5a2cf4640 1 -- start start 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c072a20 0x7fa59c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa59c072e30 0x7fa59c155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c1450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c16ad20 con 0x7fa59c143fa0 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c16aba0 con 0x7fa59c072a20 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c16aea0 con 0x7fa59c072e30 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c1450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c1450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41720/0 (socket says 172.21.15.106:41720) 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 -- 172.21.15.106:0/1911976933 learned_addr learned my addr 172.21.15.106:0/1911976933 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a0a90640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c072a20 0x7fa59c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa59bfff640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa59c072e30 0x7fa59c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 -- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa59c072e30 msgr2=0x7fa59c155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa59c072e30 0x7fa59c155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 -- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c072a20 msgr2=0x7fa59c1553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c072a20 0x7fa59c1553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 -- 172.21.15.106:0/1911976933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c15e960 con 0x7fa59c143fa0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a0a90640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c072a20 0x7fa59c1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a1291640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c1450e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa59400cad0 tx=0x7fa59400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5997fa640 1 -- 172.21.15.106:0/1911976933 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594017070 con 0x7fa59c143fa0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5997fa640 1 -- 172.21.15.106:0/1911976933 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59400ac30 con 0x7fa59c143fa0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c156170 con 0x7fa59c143fa0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5997fa640 1 -- 172.21.15.106:0/1911976933 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594012400 con 0x7fa59c143fa0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c1448e0 con 0x7fa59c143fa0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c050750 con 0x7fa59c143fa0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5997fa640 1 -- 172.21.15.106:0/1911976933 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa59401a070 con 0x7fa59c143fa0 2024-04-25T11:20:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5997fa640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa58006f430 0x7fa580071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5997fa640 1 -- 172.21.15.106:0/1911976933 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa59408ee50 con 0x7fa59c143fa0 2024-04-25T11:20:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a0a90640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa58006f430 0x7fa580071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.716+0000 7fa5a0a90640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa58006f430 0x7fa580071940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b4a60 tx=0x7fa590008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.724+0000 7fa5997fa640 1 -- 172.21.15.106:0/1911976933 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa59405b780 con 0x7fa59c143fa0 2024-04-25T11:20:05.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.876+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa59c0b08a0 con 0x7fa59c143fa0 2024-04-25T11:20:05.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5997fa640 1 -- 172.21.15.106:0/1911976933 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa59405fa80 con 0x7fa59c143fa0 2024-04-25T11:20:05.878 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:05.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa58006f430 msgr2=0x7fa580071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa58006f430 0x7fa580071940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b4a60 tx=0x7fa590008040 comp rx=0 tx=0).stop 2024-04-25T11:20:05.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 msgr2=0x7fa59c1450e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:05.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c1450e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa59400cad0 tx=0x7fa59400cf90 comp rx=0 tx=0).stop 2024-04-25T11:20:05.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 shutdown_connections 2024-04-25T11:20:05.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa58006f430 0x7fa580071940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59c143fa0 0x7fa59c1450e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa59c072e30 0x7fa59c155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 --2- 172.21.15.106:0/1911976933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c072a20 0x7fa59c1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:05.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 >> 172.21.15.106:0/1911976933 conn(0x7fa59c06c9f0 msgr2=0x7fa59c1423e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:05.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 shutdown_connections 2024-04-25T11:20:05.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:05.880+0000 7fa5a2cf4640 1 -- 172.21.15.106:0/1911976933 wait complete. 2024-04-25T11:20:05.899 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:06.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e7a3c640 1 Processor -- start 2024-04-25T11:20:06.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e7a3c640 1 -- start start 2024-04-25T11:20:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e7a3c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1e00cebd0 0x7fb1e00d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e7a3c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb1e00d1680 0x7fb1e00d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e7a3c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1e006e840 con 0x7fb1e009ba30 2024-04-25T11:20:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e7a3c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e006e610 con 0x7fb1e00cebd0 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e7a3c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e006eb20 con 0x7fb1e00d1680 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e4fd7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1e00cebd0 0x7fb1e00d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e4fd7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1e00cebd0 0x7fb1e00d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41728/0 (socket says 172.21.15.106:41728) 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e4fd7640 1 -- 172.21.15.106:0/1466507287 learned_addr learned my addr 172.21.15.106:0/1466507287 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e5fd9640 1 --2- 172.21.15.106:0/1466507287 >> v2:172.21.15.134:3300/0 conn(0x7fb1e00d1680 0x7fb1e00d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e4fd7640 1 -- 172.21.15.106:0/1466507287 >> v2:172.21.15.134:3300/0 conn(0x7fb1e00d1680 msgr2=0x7fb1e00d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e4fd7640 1 --2- 172.21.15.106:0/1466507287 >> v2:172.21.15.134:3300/0 conn(0x7fb1e00d1680 0x7fb1e00d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e4fd7640 1 -- 172.21.15.106:0/1466507287 >> v1:172.21.15.134:6789/0 conn(0x7fb1e009ba30 legacy=0x7fb1e009be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.036+0000 7fb1e4fd7640 1 -- 172.21.15.106:0/1466507287 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e006ee00 con 0x7fb1e00cebd0 2024-04-25T11:20:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e4fd7640 1 --2- 172.21.15.106:0/1466507287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00cebd0 0x7fb1e00d1140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009870 tx=0x7fb1d001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b74a4825fe81c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1d7fff640 1 -- 172.21.15.106:0/1466507287 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00b0070 con 0x7fb1e00cebd0 2024-04-25T11:20:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1d7fff640 1 -- 172.21.15.106:0/1466507287 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d00ab600 con 0x7fb1e00cebd0 2024-04-25T11:20:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1d7fff640 1 -- 172.21.15.106:0/1466507287 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00ab9d0 con 0x7fb1e00cebd0 2024-04-25T11:20:06.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/1466507287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00cebd0 msgr2=0x7fb1e00d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/1466507287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00cebd0 0x7fb1e00d1140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009870 tx=0x7fb1d001c450 comp rx=0 tx=0).stop 2024-04-25T11:20:06.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/1466507287 shutdown_connections 2024-04-25T11:20:06.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/1466507287 >> v2:172.21.15.134:3300/0 conn(0x7fb1e00d1680 0x7fb1e00d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/1466507287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00cebd0 0x7fb1e00d1140 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/1466507287 >> 172.21.15.106:0/1466507287 conn(0x7fb1e00b5060 msgr2=0x7fb1e00b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:06.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/1466507287 shutdown_connections 2024-04-25T11:20:06.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.040+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/1466507287 wait complete. 2024-04-25T11:20:06.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 Processor -- start 2024-04-25T11:20:06.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 -- start start 2024-04-25T11:20:06.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e009ba30 0x7fb1e0151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e00cebd0 0x7fb1e00ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00d1680 0x7fb1e00cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e0163620 con 0x7fb1e00d1680 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e00d25f0 con 0x7fb1e009ba30 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e01637a0 con 0x7fb1e00cebd0 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00d1680 0x7fb1e00cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00d1680 0x7fb1e00cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41734/0 (socket says 172.21.15.106:41734) 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 -- 172.21.15.106:0/3829694604 learned_addr learned my addr 172.21.15.106:0/3829694604 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e4fd7640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e00cebd0 0x7fb1e00ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e57d8640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e009ba30 0x7fb1e0151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 -- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e00cebd0 msgr2=0x7fb1e00ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e00cebd0 0x7fb1e00ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 -- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e009ba30 msgr2=0x7fb1e0151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e009ba30 0x7fb1e0151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 -- 172.21.15.106:0/3829694604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e01565a0 con 0x7fb1e00d1680 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e4fd7640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e00cebd0 0x7fb1e00ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e57d8640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e009ba30 0x7fb1e0151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e5fd9640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00d1680 0x7fb1e00cd330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc007d70 tx=0x7fb1dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1d5ffb640 1 -- 172.21.15.106:0/3829694604 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc014420 con 0x7fb1e00d1680 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1d5ffb640 1 -- 172.21.15.106:0/3829694604 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1dc019070 con 0x7fb1e00d1680 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e015b220 con 0x7fb1e00d1680 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1d5ffb640 1 -- 172.21.15.106:0/3829694604 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc014820 con 0x7fb1e00d1680 2024-04-25T11:20:06.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.044+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e015b7e0 con 0x7fb1e00d1680 2024-04-25T11:20:06.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.048+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e0050750 con 0x7fb1e00d1680 2024-04-25T11:20:06.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.048+0000 7fb1d5ffb640 1 -- 172.21.15.106:0/3829694604 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb1dc0149c0 con 0x7fb1e00d1680 2024-04-25T11:20:06.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.048+0000 7fb1d5ffb640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb1c406f400 0x7fb1c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.048+0000 7fb1d5ffb640 1 -- 172.21.15.106:0/3829694604 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb1dc08fb10 con 0x7fb1e00d1680 2024-04-25T11:20:06.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.048+0000 7fb1e57d8640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb1c406f400 0x7fb1c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.048+0000 7fb1e57d8640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb1c406f400 0x7fb1c4071910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800a830 tx=0x7fb1d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.052+0000 7fb1d5ffb640 1 -- 172.21.15.106:0/3829694604 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb1dc05ca00 con 0x7fb1e00d1680 2024-04-25T11:20:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.220+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1e0160630 con 0x7fb1e00d1680 2024-04-25T11:20:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.220+0000 7fb1d5ffb640 1 -- 172.21.15.106:0/3829694604 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fb1dc05c300 con 0x7fb1e00d1680 2024-04-25T11:20:06.220 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb1c406f400 msgr2=0x7fb1c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb1c406f400 0x7fb1c4071910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800a830 tx=0x7fb1d8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00d1680 msgr2=0x7fb1e00cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00d1680 0x7fb1e00cd330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc007d70 tx=0x7fb1dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 shutdown_connections 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb1c406f400 0x7fb1c4071910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1e00d1680 0x7fb1e00cd330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e00cebd0 0x7fb1e00ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 --2- 172.21.15.106:0/3829694604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e009ba30 0x7fb1e0151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 >> 172.21.15.106:0/3829694604 conn(0x7fb1e00b5060 msgr2=0x7fb1e00d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 shutdown_connections 2024-04-25T11:20:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.224+0000 7fb1e7a3c640 1 -- 172.21.15.106:0/3829694604 wait complete. 2024-04-25T11:20:06.243 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-25T11:20:06.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:06.243 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:06.348 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:06.348 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:20:06.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:06.348 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:20:06.357 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:20:06.357 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:06.364 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:20:06.364 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:06.413 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:06.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.564+0000 7f7d26e84640 1 Processor -- start 2024-04-25T11:20:06.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.564+0000 7f7d26e84640 1 -- start start 2024-04-25T11:20:06.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d26e84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20073460 0x7f7d200d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d26e84640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d20073870 con 0x7f7d200cef60 2024-04-25T11:20:06.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d26e84640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d200ce050 con 0x7f7d2015e0a0 2024-04-25T11:20:06.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d26e84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2015e6b0 con 0x7f7d20073460 2024-04-25T11:20:06.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7d200cef60 0x7f7d200d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37210/0 (socket says 172.21.15.106:37210) 2024-04-25T11:20:06.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1ffff640 1 -- 172.21.15.106:0/3365547995 learned_addr learned my addr 172.21.15.106:0/3365547995 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:06.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d25421640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20073460 0x7f7d200d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995781298 0 0) 0x7f7d20073870 con 0x7f7d200cef60 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d0c003530 con 0x7f7d200cef60 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 527690369 0 0) 0x7f7d0c003530 con 0x7f7d200cef60 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20073460 msgr2=0x7f7d200d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20073460 0x7f7d200d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 >> v1:172.21.15.134:6789/0 conn(0x7f7d2015e0a0 legacy=0x7f7d200ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2015e990 con 0x7f7d200cef60 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f7d08001680 con 0x7f7d200cef60 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 >> v1:172.21.15.106:6789/0 conn(0x7f7d200cef60 legacy=0x7f7d200d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20073460 0x7f7d0c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c003f70 0x7f7d0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0c004d50 0x7f7d0c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2015e6b0 con 0x7f7d0c004d50 2024-04-25T11:20:06.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c00bb60 con 0x7f7d0c003f70 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c00bd60 con 0x7f7d20073460 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d25421640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0c004d50 0x7f7d0c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1ffff640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c003f70 0x7f7d0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d24c20640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20073460 0x7f7d0c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d25421640 1 -- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20073460 msgr2=0x7f7d0c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d25421640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20073460 0x7f7d0c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d25421640 1 -- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c003f70 msgr2=0x7f7d0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d25421640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c003f70 0x7f7d0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d25421640 1 -- 172.21.15.106:0/3365547995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c003530 con 0x7f7d0c004d50 2024-04-25T11:20:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.568+0000 7f7d1ffff640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c003f70 0x7f7d0c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:06.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d25421640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0c004d50 0x7f7d0c005160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009fc0 tx=0x7f7d1801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db66b6187ae29e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180b2070 con 0x7f7d0c004d50 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d180a5cb0 con 0x7f7d0c004d50 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d1f7fe640 1 -- 172.21.15.106:0/3365547995 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18009c10 con 0x7f7d0c004d50 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d26e84640 1 -- 172.21.15.106:0/3365547995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0c004d50 msgr2=0x7f7d0c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d26e84640 1 --2- 172.21.15.106:0/3365547995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0c004d50 0x7f7d0c005160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009fc0 tx=0x7f7d1801cca0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d25421640 1 -- 172.21.15.106:0/3365547995 reap_dead start 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d26e84640 1 -- 172.21.15.106:0/3365547995 shutdown_connections 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d26e84640 1 -- 172.21.15.106:0/3365547995 >> 172.21.15.106:0/3365547995 conn(0x7f7d2006c7c0 msgr2=0x7f7d2006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d26e84640 1 -- 172.21.15.106:0/3365547995 shutdown_connections 2024-04-25T11:20:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.572+0000 7f7d26e84640 1 -- 172.21.15.106:0/3365547995 wait complete. 2024-04-25T11:20:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 Processor -- start 2024-04-25T11:20:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 -- start start 2024-04-25T11:20:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c004d50 0x7f7d201f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20147250 0x7f7d200ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d200aa490 0x7f7d201f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d200d06e0 con 0x7f7d200aa490 2024-04-25T11:20:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2015ef50 con 0x7f7d0c004d50 2024-04-25T11:20:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d200d0860 con 0x7f7d20147250 2024-04-25T11:20:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d200aa490 0x7f7d201f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d200aa490 0x7f7d201f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41748/0 (socket says 172.21.15.106:41748) 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 -- 172.21.15.106:0/123607465 learned_addr learned my addr 172.21.15.106:0/123607465 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d24c20640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c004d50 0x7f7d201f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d1ffff640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20147250 0x7f7d200ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 -- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20147250 msgr2=0x7f7d200ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20147250 0x7f7d200ab790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 -- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c004d50 msgr2=0x7f7d201f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c004d50 0x7f7d201f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 -- 172.21.15.106:0/123607465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d201f7b20 con 0x7f7d200aa490 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d24c20640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c004d50 0x7f7d201f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d25421640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d200aa490 0x7f7d201f4540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7d180b0040 tx=0x7f7d18009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d1d7fa640 1 -- 172.21.15.106:0/123607465 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180bb020 con 0x7f7d200aa490 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d1d7fa640 1 -- 172.21.15.106:0/123607465 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d180a5ab0 con 0x7f7d200aa490 2024-04-25T11:20:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d201f7e00 con 0x7f7d200aa490 2024-04-25T11:20:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d1d7fa640 1 -- 172.21.15.106:0/123607465 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180b2040 con 0x7f7d200aa490 2024-04-25T11:20:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d201f3d40 con 0x7f7d200aa490 2024-04-25T11:20:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.576+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d201f7840 con 0x7f7d200aa490 2024-04-25T11:20:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.580+0000 7f7d1d7fa640 1 -- 172.21.15.106:0/123607465 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7d180b6410 con 0x7f7d200aa490 2024-04-25T11:20:06.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.580+0000 7f7d1d7fa640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7d0006f400 0x7f7d00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.580+0000 7f7d24c20640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7d0006f400 0x7f7d00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.580+0000 7f7d1d7fa640 1 -- 172.21.15.106:0/123607465 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f7d18129a20 con 0x7f7d200aa490 2024-04-25T11:20:06.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.580+0000 7f7d24c20640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7d0006f400 0x7f7d00071910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7d100016a0 tx=0x7f7d10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.584+0000 7f7d1d7fa640 1 -- 172.21.15.106:0/123607465 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d180c5020 con 0x7f7d200aa490 2024-04-25T11:20:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.732+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d200abcd0 con 0x7f7d200aa490 2024-04-25T11:20:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d1d7fa640 1 -- 172.21.15.106:0/123607465 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16834 (secure 0 0 0) 0x7f7d180169b0 con 0x7f7d200aa490 2024-04-25T11:20:06.735 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:06.735 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":21,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:06.229239+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7d0006f400 msgr2=0x7f7d00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7d0006f400 0x7f7d00071910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7d100016a0 tx=0x7f7d10009040 comp rx=0 tx=0).stop 2024-04-25T11:20:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d200aa490 msgr2=0x7f7d201f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d200aa490 0x7f7d201f4540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7d180b0040 tx=0x7f7d18009750 comp rx=0 tx=0).stop 2024-04-25T11:20:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 shutdown_connections 2024-04-25T11:20:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7d0006f400 0x7f7d00071910 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d200aa490 0x7f7d201f4540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d20147250 0x7f7d200ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 --2- 172.21.15.106:0/123607465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c004d50 0x7f7d201f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 >> 172.21.15.106:0/123607465 conn(0x7f7d2006c7c0 msgr2=0x7f7d200d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:06.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 shutdown_connections 2024-04-25T11:20:06.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.736+0000 7f7d26e84640 1 -- 172.21.15.106:0/123607465 wait complete. 2024-04-25T11:20:06.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:06.756 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:06.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2529065640 1 Processor -- start 2024-04-25T11:20:06.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2529065640 1 -- start start 2024-04-25T11:20:06.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2529065640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f25240d0cf0 0x7f252406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2529065640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25240b68e0 con 0x7f252415d230 2024-04-25T11:20:06.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2529065640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2524161210 con 0x7f25240d1740 2024-04-25T11:20:06.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2529065640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f25240b6290 con 0x7f25240d0cf0 2024-04-25T11:20:06.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f252359e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f252415d230 0x7f252415d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37226/0 (socket says 172.21.15.106:37226) 2024-04-25T11:20:06.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f252359e640 1 -- 172.21.15.106:0/3414366950 learned_addr learned my addr 172.21.15.106:0/3414366950 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:06.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2522d9d640 1 --2- 172.21.15.106:0/3414366950 >> v2:172.21.15.134:3300/0 conn(0x7f25240d0cf0 0x7f252406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708722115 0 0) 0x7f25240b68e0 con 0x7f252415d230 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.908+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2510003530 con 0x7f252415d230 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2006744223 0 0) 0x7f2510003530 con 0x7f252415d230 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 >> v2:172.21.15.134:3300/0 conn(0x7f25240d0cf0 msgr2=0x7f252406ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 --2- 172.21.15.106:0/3414366950 >> v2:172.21.15.134:3300/0 conn(0x7f25240d0cf0 0x7f252406ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 >> v1:172.21.15.134:6789/0 conn(0x7f25240d1740 legacy=0x7f25240d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25240b6570 con 0x7f252415d230 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f251c001a00 con 0x7f252415d230 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 >> v1:172.21.15.106:6789/0 conn(0x7f252415d230 legacy=0x7f252415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25100039c0 0x7f2510003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2510004810 0x7f2510004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2510005360 0x7f251000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25240b6290 con 0x7f2510004810 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f251000c290 con 0x7f25100039c0 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f251000c490 con 0x7f2510005360 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252259c640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2510004810 0x7f2510004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2522d9d640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25100039c0 0x7f2510003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252359e640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2510005360 0x7f251000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252259c640 1 -- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2510005360 msgr2=0x7f251000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252259c640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2510005360 0x7f251000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252259c640 1 -- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25100039c0 msgr2=0x7f2510003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252259c640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25100039c0 0x7f2510003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252259c640 1 -- 172.21.15.106:0/3414366950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2510003530 con 0x7f2510004810 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252359e640 1 -- 172.21.15.106:0/3414366950 reap_dead start 2024-04-25T11:20:06.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2522d9d640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25100039c0 0x7f2510003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:06.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f252259c640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2510004810 0x7f2510004c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2514001750 tx=0x7f251401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e9426e6a6cc800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25140013a0 con 0x7f2510004810 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25140052f0 con 0x7f2510004810 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2521d9b640 1 -- 172.21.15.106:0/3414366950 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514003840 con 0x7f2510004810 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2529065640 1 -- 172.21.15.106:0/3414366950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2510004810 msgr2=0x7f2510004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2529065640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2510004810 0x7f2510004c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2514001750 tx=0x7f251401c3c0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2529065640 1 -- 172.21.15.106:0/3414366950 shutdown_connections 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2529065640 1 --2- 172.21.15.106:0/3414366950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2510004810 0x7f2510004c20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2529065640 1 -- 172.21.15.106:0/3414366950 >> 172.21.15.106:0/3414366950 conn(0x7f252406c9f0 msgr2=0x7f252406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.912+0000 7f2529065640 1 -- 172.21.15.106:0/3414366950 shutdown_connections 2024-04-25T11:20:06.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 -- 172.21.15.106:0/3414366950 wait complete. 2024-04-25T11:20:06.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 Processor -- start 2024-04-25T11:20:06.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 -- start start 2024-04-25T11:20:06.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510004810 0x7f2524148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25241f6fb0 0x7f2524155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524156090 0x7f25241f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25240d1f60 con 0x7f2524156090 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f252415cbc0 con 0x7f2510004810 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2529065640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2524154900 con 0x7f25241f6fb0 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524156090 0x7f25241f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524156090 0x7f25241f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41776/0 (socket says 172.21.15.106:41776) 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 -- 172.21.15.106:0/600348465 learned_addr learned my addr 172.21.15.106:0/600348465 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f2522d9d640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510004810 0x7f2524148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252259c640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25241f6fb0 0x7f2524155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 -- 172.21.15.106:0/600348465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25241f6fb0 msgr2=0x7f2524155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25241f6fb0 0x7f2524155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 -- 172.21.15.106:0/600348465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510004810 msgr2=0x7f2524148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:06.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510004810 0x7f2524148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:06.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 -- 172.21.15.106:0/600348465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25241564a0 con 0x7f2524156090 2024-04-25T11:20:06.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.916+0000 7f252359e640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524156090 0x7f25241f9b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f251c000f90 tx=0x7f251c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f251b7fe640 1 -- 172.21.15.106:0/600348465 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c00ebc0 con 0x7f2524156090 2024-04-25T11:20:06.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f251b7fe640 1 -- 172.21.15.106:0/600348465 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251c018070 con 0x7f2524156090 2024-04-25T11:20:06.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f251b7fe640 1 -- 172.21.15.106:0/600348465 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c013420 con 0x7f2524156090 2024-04-25T11:20:06.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2524149280 con 0x7f2524156090 2024-04-25T11:20:06.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2524156fe0 con 0x7f2524156090 2024-04-25T11:20:06.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2524156a70 con 0x7f2524156090 2024-04-25T11:20:06.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f251b7fe640 1 -- 172.21.15.106:0/600348465 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f251c0135c0 con 0x7f2524156090 2024-04-25T11:20:06.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f251b7fe640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f250406f400 0x7f2504071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:06.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f251b7fe640 1 -- 172.21.15.106:0/600348465 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f251c08f640 con 0x7f2524156090 2024-04-25T11:20:06.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f2522d9d640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f250406f400 0x7f2504071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:06.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.920+0000 7f2522d9d640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f250406f400 0x7f2504071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f252406afe0 tx=0x7f250c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:06.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:06.924+0000 7f251b7fe640 1 -- 172.21.15.106:0/600348465 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25241564a0 con 0x7f2524156090 2024-04-25T11:20:07.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.088+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2524050750 con 0x7f2524156090 2024-04-25T11:20:07.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.088+0000 7f251b7fe640 1 -- 172.21.15.106:0/600348465 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f251c05c4b0 con 0x7f2524156090 2024-04-25T11:20:07.088 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:07.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f250406f400 msgr2=0x7f2504071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f250406f400 0x7f2504071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f252406afe0 tx=0x7f250c008040 comp rx=0 tx=0).stop 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524156090 msgr2=0x7f25241f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524156090 0x7f25241f9b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f251c000f90 tx=0x7f251c007bc0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 shutdown_connections 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f250406f400 0x7f2504071910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524156090 0x7f25241f9b80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25241f6fb0 0x7f2524155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 --2- 172.21.15.106:0/600348465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510004810 0x7f2524148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 >> 172.21.15.106:0/600348465 conn(0x7f252406c9f0 msgr2=0x7f25240b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 shutdown_connections 2024-04-25T11:20:07.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.092+0000 7f2529065640 1 -- 172.21.15.106:0/600348465 wait complete. 2024-04-25T11:20:07.111 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:07.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.272+0000 7fbe70401640 1 Processor -- start 2024-04-25T11:20:07.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.272+0000 7fbe70401640 1 -- start start 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe70401640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbe680b7cb0 0x7fbe680cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe70401640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680cef30 0x7fbe680cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe70401640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe680b4d70 con 0x7fbe680b53b0 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe70401640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbe680cdb20 con 0x7fbe680b7cb0 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe70401640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe680b3c70 con 0x7fbe680cef30 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6e19d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbe680b7cb0 0x7fbe680cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6e99e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbe680b53b0 0x7fbe680b57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37236/0 (socket says 172.21.15.106:37236) 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6e99e640 1 -- 172.21.15.106:0/2331475628 learned_addr learned my addr 172.21.15.106:0/2331475628 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6e19d640 1 -- 172.21.15.106:0/2331475628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680cef30 msgr2=0x7fbe680cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:07.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6e19d640 1 --2- 172.21.15.106:0/2331475628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680cef30 0x7fbe680cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6e19d640 1 -- 172.21.15.106:0/2331475628 >> v1:172.21.15.106:6789/0 conn(0x7fbe680b53b0 legacy=0x7fbe680b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6e19d640 1 -- 172.21.15.106:0/2331475628 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe680ce4f0 con 0x7fbe680b7cb0 2024-04-25T11:20:07.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6d19b640 1 -- 172.21.15.106:0/2331475628 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3264424102 0 0) 0x7fbe680b4d70 con 0x7fbe680b53b0 2024-04-25T11:20:07.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6e19d640 1 --2- 172.21.15.106:0/2331475628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680b7cb0 0x7fbe680cdfb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbe640097c0 tx=0x7fbe6401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f31ad09afe8e703 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6d19b640 1 -- 172.21.15.106:0/2331475628 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe640b0070 con 0x7fbe680b7cb0 2024-04-25T11:20:07.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6d19b640 1 -- 172.21.15.106:0/2331475628 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe640ab600 con 0x7fbe680b7cb0 2024-04-25T11:20:07.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.276+0000 7fbe6d19b640 1 -- 172.21.15.106:0/2331475628 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe640ab9d0 con 0x7fbe680b7cb0 2024-04-25T11:20:07.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- 172.21.15.106:0/2331475628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680b7cb0 msgr2=0x7fbe680cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 --2- 172.21.15.106:0/2331475628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680b7cb0 0x7fbe680cdfb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbe640097c0 tx=0x7fbe6401c4a0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- 172.21.15.106:0/2331475628 shutdown_connections 2024-04-25T11:20:07.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 --2- 172.21.15.106:0/2331475628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680cef30 0x7fbe680cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 --2- 172.21.15.106:0/2331475628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680b7cb0 0x7fbe680cdfb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- 172.21.15.106:0/2331475628 >> 172.21.15.106:0/2331475628 conn(0x7fbe680b40b0 msgr2=0x7fbe680b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:07.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- 172.21.15.106:0/2331475628 shutdown_connections 2024-04-25T11:20:07.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- 172.21.15.106:0/2331475628 wait complete. 2024-04-25T11:20:07.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 Processor -- start 2024-04-25T11:20:07.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- start start 2024-04-25T11:20:07.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe680b53b0 0x7fbe680cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680b7cb0 0x7fbe68152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680cef30 0x7fbe68158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6806d1d0 con 0x7fbe680cef30 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6806d050 con 0x7fbe680b53b0 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe70401640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6806d350 con 0x7fbe680b7cb0 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe6e99e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680cef30 0x7fbe68158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe6e99e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680cef30 0x7fbe68158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41800/0 (socket says 172.21.15.106:41800) 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.280+0000 7fbe6e99e640 1 -- 172.21.15.106:0/1756623298 learned_addr learned my addr 172.21.15.106:0/1756623298 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e19d640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe680b53b0 0x7fbe680cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6d99c640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680b7cb0 0x7fbe68152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e99e640 1 -- 172.21.15.106:0/1756623298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680b7cb0 msgr2=0x7fbe68152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e99e640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680b7cb0 0x7fbe68152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e99e640 1 -- 172.21.15.106:0/1756623298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe680b53b0 msgr2=0x7fbe680cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e99e640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe680b53b0 0x7fbe680cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e99e640 1 -- 172.21.15.106:0/1756623298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6809df00 con 0x7fbe680cef30 2024-04-25T11:20:07.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e99e640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680cef30 0x7fbe68158830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000c700 tx=0x7fbe6000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe5effd640 1 -- 172.21.15.106:0/1756623298 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60018070 con 0x7fbe680cef30 2024-04-25T11:20:07.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe5effd640 1 -- 172.21.15.106:0/1756623298 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe6000ce10 con 0x7fbe680cef30 2024-04-25T11:20:07.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe5effd640 1 -- 172.21.15.106:0/1756623298 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6000a880 con 0x7fbe680cef30 2024-04-25T11:20:07.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6809e1e0 con 0x7fbe680cef30 2024-04-25T11:20:07.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6809e4c0 con 0x7fbe680cef30 2024-04-25T11:20:07.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6809dc70 con 0x7fbe680cef30 2024-04-25T11:20:07.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe5effd640 1 -- 172.21.15.106:0/1756623298 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbe6000aa20 con 0x7fbe680cef30 2024-04-25T11:20:07.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe5effd640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbe4c06f400 0x7fbe4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e19d640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbe4c06f400 0x7fbe4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe5effd640 1 -- 172.21.15.106:0/1756623298 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbe6008e7d0 con 0x7fbe680cef30 2024-04-25T11:20:07.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.284+0000 7fbe6e19d640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbe4c06f400 0x7fbe4c071910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe640ad000 tx=0x7fbe640a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.288+0000 7fbe5effd640 1 -- 172.21.15.106:0/1756623298 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe6005b760 con 0x7fbe680cef30 2024-04-25T11:20:07.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.452+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe68050750 con 0x7fbe680cef30 2024-04-25T11:20:07.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.452+0000 7fbe5effd640 1 -- 172.21.15.106:0/1756623298 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fbe6005b060 con 0x7fbe680cef30 2024-04-25T11:20:07.452 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:07.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbe4c06f400 msgr2=0x7fbe4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbe4c06f400 0x7fbe4c071910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe640ad000 tx=0x7fbe640a6000 comp rx=0 tx=0).stop 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680cef30 msgr2=0x7fbe68158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680cef30 0x7fbe68158830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000c700 tx=0x7fbe6000cbc0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 shutdown_connections 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbe4c06f400 0x7fbe4c071910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe680cef30 0x7fbe68158830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe680b7cb0 0x7fbe68152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 --2- 172.21.15.106:0/1756623298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe680b53b0 0x7fbe680cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 >> 172.21.15.106:0/1756623298 conn(0x7fbe680b40b0 msgr2=0x7fbe680c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 shutdown_connections 2024-04-25T11:20:07.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.456+0000 7fbe70401640 1 -- 172.21.15.106:0/1756623298 wait complete. 2024-04-25T11:20:07.474 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22bd205640 1 Processor -- start 2024-04-25T11:20:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22bd205640 1 -- start start 2024-04-25T11:20:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22bd205640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f22b8143f10 0x7f22b80d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22bd205640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22b80d0fa0 con 0x7f22b80d1740 2024-04-25T11:20:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22bd205640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22b81618c0 con 0x7f22b8072a20 2024-04-25T11:20:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22bd205640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f22b8144320 con 0x7f22b8143f10 2024-04-25T11:20:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22b6d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f22b8143f10 0x7f22b80d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22b659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f22b80d1740 0x7f22b80d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37244/0 (socket says 172.21.15.106:37244) 2024-04-25T11:20:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22b659c640 1 -- 172.21.15.106:0/2848088196 learned_addr learned my addr 172.21.15.106:0/2848088196 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22b6d9d640 1 -- 172.21.15.106:0/2848088196 >> v1:172.21.15.134:6789/0 conn(0x7f22b8072a20 legacy=0x7f22b8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22b6d9d640 1 -- 172.21.15.106:0/2848088196 >> v1:172.21.15.106:6789/0 conn(0x7f22b80d1740 legacy=0x7f22b80d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22b6d9d640 1 -- 172.21.15.106:0/2848088196 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b8161ba0 con 0x7f22b8143f10 2024-04-25T11:20:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.624+0000 7f22b5d9b640 1 -- 172.21.15.106:0/2848088196 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3385414461 0 0) 0x7f22b80d0fa0 con 0x7f22b80d1740 2024-04-25T11:20:07.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22b6d9d640 1 --2- 172.21.15.106:0/2848088196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80d05c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22a80097c0 tx=0x7f22a801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c45a98cd209270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22b5d9b640 1 -- 172.21.15.106:0/2848088196 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a80b0070 con 0x7f22b8143f10 2024-04-25T11:20:07.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22b5d9b640 1 -- 172.21.15.106:0/2848088196 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a80ab600 con 0x7f22b8143f10 2024-04-25T11:20:07.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22b5d9b640 1 -- 172.21.15.106:0/2848088196 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a80ab9d0 con 0x7f22b8143f10 2024-04-25T11:20:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22bd205640 1 -- 172.21.15.106:0/2848088196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 msgr2=0x7f22b80d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22bd205640 1 --2- 172.21.15.106:0/2848088196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80d05c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22a80097c0 tx=0x7f22a801c340 comp rx=0 tx=0).stop 2024-04-25T11:20:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22bd205640 1 -- 172.21.15.106:0/2848088196 shutdown_connections 2024-04-25T11:20:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22bd205640 1 --2- 172.21.15.106:0/2848088196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80d05c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22bd205640 1 -- 172.21.15.106:0/2848088196 >> 172.21.15.106:0/2848088196 conn(0x7f22b806c9f0 msgr2=0x7f22b806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22bd205640 1 -- 172.21.15.106:0/2848088196 shutdown_connections 2024-04-25T11:20:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.628+0000 7f22bd205640 1 -- 172.21.15.106:0/2848088196 wait complete. 2024-04-25T11:20:07.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 Processor -- start 2024-04-25T11:20:07.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 -- start start 2024-04-25T11:20:07.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8072a20 0x7f22b815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22b80d1740 0x7f22b80af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8073950 con 0x7f22b8143f10 2024-04-25T11:20:07.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f22b81625b0 con 0x7f22b8072a20 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f22b80d0d10 con 0x7f22b80d1740 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41816/0 (socket says 172.21.15.106:41816) 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 -- 172.21.15.106:0/2198151956 learned_addr learned my addr 172.21.15.106:0/2198151956 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b659c640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22b80d1740 0x7f22b80af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b6d9d640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8072a20 0x7f22b815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 -- 172.21.15.106:0/2198151956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22b80d1740 msgr2=0x7f22b80af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22b80d1740 0x7f22b80af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 -- 172.21.15.106:0/2198151956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8072a20 msgr2=0x7f22b815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8072a20 0x7f22b815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 -- 172.21.15.106:0/2198151956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b80cf050 con 0x7f22b8143f10 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22b759e640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80afdf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f22b000be90 tx=0x7f22b000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f229f7fe640 1 -- 172.21.15.106:0/2198151956 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0019070 con 0x7f22b8143f10 2024-04-25T11:20:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f229f7fe640 1 -- 172.21.15.106:0/2198151956 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b0014440 con 0x7f22b8143f10 2024-04-25T11:20:07.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b80aeaf0 con 0x7f22b8143f10 2024-04-25T11:20:07.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f229f7fe640 1 -- 172.21.15.106:0/2198151956 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0014870 con 0x7f22b8143f10 2024-04-25T11:20:07.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b80af0b0 con 0x7f22b8143f10 2024-04-25T11:20:07.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.632+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b8050750 con 0x7f22b8143f10 2024-04-25T11:20:07.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.636+0000 7f229f7fe640 1 -- 172.21.15.106:0/2198151956 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22b001c070 con 0x7f22b8143f10 2024-04-25T11:20:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.636+0000 7f229f7fe640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22a406f430 0x7f22a4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.636+0000 7f229f7fe640 1 -- 172.21.15.106:0/2198151956 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f22b008f370 con 0x7f22b8143f10 2024-04-25T11:20:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.636+0000 7f22b6d9d640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22a406f430 0x7f22a4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.636+0000 7f22b6d9d640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22a406f430 0x7f22a4071940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f22a80ad000 tx=0x7f22a80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.640+0000 7f229f7fe640 1 -- 172.21.15.106:0/2198151956 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22b80cf050 con 0x7f22b8143f10 2024-04-25T11:20:07.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.796+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22b80b2c30 con 0x7f22b8143f10 2024-04-25T11:20:07.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.796+0000 7f229f7fe640 1 -- 172.21.15.106:0/2198151956 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f22b005c270 con 0x7f22b8143f10 2024-04-25T11:20:07.796 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:07.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22a406f430 msgr2=0x7f22a4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22a406f430 0x7f22a4071940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f22a80ad000 tx=0x7f22a80a6000 comp rx=0 tx=0).stop 2024-04-25T11:20:07.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 msgr2=0x7f22b80afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80afdf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f22b000be90 tx=0x7f22b000bf80 comp rx=0 tx=0).stop 2024-04-25T11:20:07.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 shutdown_connections 2024-04-25T11:20:07.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22a406f430 0x7f22a4071940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22b8143f10 0x7f22b80afdf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22b80d1740 0x7f22b80af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 --2- 172.21.15.106:0/2198151956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8072a20 0x7f22b815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 >> 172.21.15.106:0/2198151956 conn(0x7f22b806c9f0 msgr2=0x7f22b80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:07.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 shutdown_connections 2024-04-25T11:20:07.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.800+0000 7f22bd205640 1 -- 172.21.15.106:0/2198151956 wait complete. 2024-04-25T11:20:07.816 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:07.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.964+0000 7efdbcb59640 1 Processor -- start 2024-04-25T11:20:07.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.964+0000 7efdbcb59640 1 -- start start 2024-04-25T11:20:07.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdbcb59640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efdb8166ab0 0x7efdb8168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdbcb59640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb80b5730 con 0x7efdb80cf010 2024-04-25T11:20:07.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdbcb59640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb80b7c50 con 0x7efdb80b5320 2024-04-25T11:20:07.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdbcb59640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efdb80ceb80 con 0x7efdb8166ab0 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdadd9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efdb80cf010 0x7efdb80cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37248/0 (socket says 172.21.15.106:37248) 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdadd9b640 1 -- 172.21.15.106:0/901713020 learned_addr learned my addr 172.21.15.106:0/901713020 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdb6d9d640 1 --2- 172.21.15.106:0/901713020 >> v2:172.21.15.106:3300/0 conn(0x7efdb8166ab0 0x7efdb8168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdb5d9b640 1 -- 172.21.15.106:0/901713020 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114038244 0 0) 0x7efda0001120 con 0x7efdb80cf010 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdb5d9b640 1 -- 172.21.15.106:0/901713020 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efda4003530 con 0x7efdb80cf010 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdb6d9d640 1 -- 172.21.15.106:0/901713020 >> v1:172.21.15.134:6789/0 conn(0x7efdb80b5320 legacy=0x7efdb815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdb6d9d640 1 -- 172.21.15.106:0/901713020 >> v1:172.21.15.106:6789/0 conn(0x7efdb80cf010 legacy=0x7efdb80cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdb5d9b640 1 -- 172.21.15.106:0/901713020 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3177318446 0 0) 0x7efda4003530 con 0x7efdb80cf010 2024-04-25T11:20:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.968+0000 7efdb6d9d640 1 -- 172.21.15.106:0/901713020 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb80b7970 con 0x7efdb8166ab0 2024-04-25T11:20:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdb6d9d640 1 --2- 172.21.15.106:0/901713020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb8166ab0 0x7efdb8168ed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009d90 tx=0x7efdb001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5bb6f4e1251038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdb5d9b640 1 -- 172.21.15.106:0/901713020 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb00b2070 con 0x7efdb8166ab0 2024-04-25T11:20:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdb5d9b640 1 -- 172.21.15.106:0/901713020 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb00a55e0 con 0x7efdb8166ab0 2024-04-25T11:20:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdb5d9b640 1 -- 172.21.15.106:0/901713020 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0001850 con 0x7efdb8166ab0 2024-04-25T11:20:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 -- 172.21.15.106:0/901713020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb8166ab0 msgr2=0x7efdb8168ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 --2- 172.21.15.106:0/901713020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb8166ab0 0x7efdb8168ed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009d90 tx=0x7efdb001c890 comp rx=0 tx=0).stop 2024-04-25T11:20:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 -- 172.21.15.106:0/901713020 shutdown_connections 2024-04-25T11:20:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 --2- 172.21.15.106:0/901713020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb8166ab0 0x7efdb8168ed0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 -- 172.21.15.106:0/901713020 >> 172.21.15.106:0/901713020 conn(0x7efdb80b4020 msgr2=0x7efdb80b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 -- 172.21.15.106:0/901713020 shutdown_connections 2024-04-25T11:20:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 -- 172.21.15.106:0/901713020 wait complete. 2024-04-25T11:20:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 Processor -- start 2024-04-25T11:20:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.972+0000 7efdbcb59640 1 -- start start 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb80b5320 0x7efdb8154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdb80cf010 0x7efdb815fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efdb8166ab0 0x7efdb8160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efdb80cfcb0 con 0x7efdb80b5320 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efdb80cfb30 con 0x7efdb80cf010 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efdb816bc30 con 0x7efdb8166ab0 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb80b5320 0x7efdb8154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb80b5320 0x7efdb8154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41830/0 (socket says 172.21.15.106:41830) 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 -- 172.21.15.106:0/1059452039 learned_addr learned my addr 172.21.15.106:0/1059452039 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdadd9b640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdb80cf010 0x7efdb815fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 -- 172.21.15.106:0/1059452039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efdb8166ab0 msgr2=0x7efdb8160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efdb8166ab0 0x7efdb8160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 -- 172.21.15.106:0/1059452039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdb80cf010 msgr2=0x7efdb815fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdb80cf010 0x7efdb815fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 -- 172.21.15.106:0/1059452039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb80af5e0 con 0x7efdb80b5320 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdb659c640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb80b5320 0x7efdb8154130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efda800c3c0 tx=0x7efda800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdaf7fe640 1 -- 172.21.15.106:0/1059452039 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8017070 con 0x7efdb80b5320 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdaf7fe640 1 -- 172.21.15.106:0/1059452039 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda80052d0 con 0x7efdb80b5320 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdaf7fe640 1 -- 172.21.15.106:0/1059452039 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8003840 con 0x7efdb80b5320 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb80af8c0 con 0x7efdb80b5320 2024-04-25T11:20:07.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb80afe80 con 0x7efdb80b5320 2024-04-25T11:20:07.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb8050750 con 0x7efdb80b5320 2024-04-25T11:20:07.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdaf7fe640 1 -- 172.21.15.106:0/1059452039 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efda801a070 con 0x7efdb80b5320 2024-04-25T11:20:07.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdaf7fe640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efd9806f430 0x7efd98071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:07.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdadd9b640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efd9806f430 0x7efd98071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:07.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdaf7fe640 1 -- 172.21.15.106:0/1059452039 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7efda808e610 con 0x7efdb80b5320 2024-04-25T11:20:07.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.976+0000 7efdadd9b640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efd9806f430 0x7efd98071940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efda0005890 tx=0x7efda0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:07.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:07.980+0000 7efdaf7fe640 1 -- 172.21.15.106:0/1059452039 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efda8005480 con 0x7efdb80b5320 2024-04-25T11:20:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.140+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efdb806eb80 con 0x7efdb80b5320 2024-04-25T11:20:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdaf7fe640 1 -- 172.21.15.106:0/1059452039 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7efda805b480 con 0x7efdb80b5320 2024-04-25T11:20:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:08.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efd9806f430 msgr2=0x7efd98071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efd9806f430 0x7efd98071940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efda0005890 tx=0x7efda0009040 comp rx=0 tx=0).stop 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb80b5320 msgr2=0x7efdb8154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb80b5320 0x7efdb8154130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efda800c3c0 tx=0x7efda800c880 comp rx=0 tx=0).stop 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 shutdown_connections 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efdb8166ab0 0x7efdb8160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdb80cf010 0x7efdb815fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efd9806f430 0x7efd98071940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 --2- 172.21.15.106:0/1059452039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdb80b5320 0x7efdb8154130 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 >> 172.21.15.106:0/1059452039 conn(0x7efdb80b4020 msgr2=0x7efdb80cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 shutdown_connections 2024-04-25T11:20:08.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.144+0000 7efdbcb59640 1 -- 172.21.15.106:0/1059452039 wait complete. 2024-04-25T11:20:08.162 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:20:08.162 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:08.169 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:08.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bb1904640 1 Processor -- start 2024-04-25T11:20:08.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bb1904640 1 -- start start 2024-04-25T11:20:08.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bb1904640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac0cf910 0x7f4bac0d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bb1904640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bac0cfd20 con 0x7f4bac0ce140 2024-04-25T11:20:08.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bb1904640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bac0b2730 con 0x7f4bac0b2e90 2024-04-25T11:20:08.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bb1904640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac0d0c70 con 0x7f4bac0cf910 2024-04-25T11:20:08.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bab7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4bac0ce140 0x7f4bac0ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37256/0 (socket says 172.21.15.106:37256) 2024-04-25T11:20:08.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bab7fe640 1 -- 172.21.15.106:0/1570853296 learned_addr learned my addr 172.21.15.106:0/1570853296 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:08.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4baaffd640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac0cf910 0x7f4bac0d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1970443791 0 0) 0x7f4bac0cfd20 con 0x7f4bac0ce140 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b94003530 con 0x7f4bac0ce140 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2109386810 0 0) 0x7f4b94003530 con 0x7f4bac0ce140 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac0cf910 msgr2=0x7f4bac0d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac0cf910 0x7f4bac0d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 >> v1:172.21.15.134:6789/0 conn(0x7f4bac0b2e90 legacy=0x7f4bac0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac0cf3f0 con 0x7f4bac0ce140 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f4ba4001a00 con 0x7f4bac0ce140 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 >> v1:172.21.15.106:6789/0 conn(0x7f4bac0ce140 legacy=0x7f4bac0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac0cf910 0x7f4b940039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b94003f70 0x7f4b94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b94004d50 0x7f4b94005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0d0c70 con 0x7f4b94004d50 2024-04-25T11:20:08.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9400bb60 con 0x7f4b94003f70 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9400bd60 con 0x7f4bac0cf910 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4bab7fe640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b94004d50 0x7f4b94005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4baaffd640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac0cf910 0x7f4b940039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.320+0000 7f4baa7fc640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b94003f70 0x7f4b94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bab7fe640 1 -- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac0cf910 msgr2=0x7f4b940039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bab7fe640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac0cf910 0x7f4b940039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bab7fe640 1 -- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b94003f70 msgr2=0x7f4b94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bab7fe640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b94003f70 0x7f4b94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bab7fe640 1 -- 172.21.15.106:0/1570853296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b94003530 con 0x7f4b94004d50 2024-04-25T11:20:08.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bab7fe640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b94004d50 0x7f4b94005160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0ca5b0 tx=0x7f4ba401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96cd2bd494fb543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:08.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400a4a0 con 0x7f4b94004d50 2024-04-25T11:20:08.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba40ad650 con 0x7f4b94004d50 2024-04-25T11:20:08.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4ba9ffb640 1 -- 172.21.15.106:0/1570853296 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40ada40 con 0x7f4b94004d50 2024-04-25T11:20:08.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bb1904640 1 -- 172.21.15.106:0/1570853296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b94004d50 msgr2=0x7f4b94005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bb1904640 1 --2- 172.21.15.106:0/1570853296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b94004d50 0x7f4b94005160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0ca5b0 tx=0x7f4ba401d360 comp rx=0 tx=0).stop 2024-04-25T11:20:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bab7fe640 1 -- 172.21.15.106:0/1570853296 reap_dead start 2024-04-25T11:20:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bb1904640 1 -- 172.21.15.106:0/1570853296 shutdown_connections 2024-04-25T11:20:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bb1904640 1 -- 172.21.15.106:0/1570853296 >> 172.21.15.106:0/1570853296 conn(0x7f4bac0cffb0 msgr2=0x7f4bac0d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bb1904640 1 -- 172.21.15.106:0/1570853296 shutdown_connections 2024-04-25T11:20:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.324+0000 7f4bb1904640 1 -- 172.21.15.106:0/1570853296 wait complete. 2024-04-25T11:20:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 Processor -- start 2024-04-25T11:20:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 -- start start 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bac15bd50 0x7f4bac0b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bac0b74d0 0x7f4bac160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac160b50 0x7f4bac0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0b88d0 con 0x7f4bac15bd50 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0b8750 con 0x7f4bac0b74d0 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac0b8eb0 con 0x7f4bac160b50 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bac15bd50 0x7f4bac0b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bac15bd50 0x7f4bac0b6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41858/0 (socket says 172.21.15.106:41858) 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 -- 172.21.15.106:0/925520813 learned_addr learned my addr 172.21.15.106:0/925520813 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baa7fc640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bac0b74d0 0x7f4bac160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bab7fe640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac160b50 0x7f4bac0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 -- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac160b50 msgr2=0x7f4bac0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac160b50 0x7f4bac0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 -- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bac0b74d0 msgr2=0x7f4bac160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bac0b74d0 0x7f4bac160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 -- 172.21.15.106:0/925520813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac0b78e0 con 0x7f4bac15bd50 2024-04-25T11:20:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baa7fc640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bac0b74d0 0x7f4bac160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bab7fe640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac160b50 0x7f4bac0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4baaffd640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bac15bd50 0x7f4bac0b6f90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c011600 tx=0x7f4b9c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4b93fff640 1 -- 172.21.15.106:0/925520813 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c019070 con 0x7f4bac15bd50 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4b93fff640 1 -- 172.21.15.106:0/925520813 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9c005db0 con 0x7f4bac15bd50 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bac06d700 con 0x7f4bac15bd50 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4b93fff640 1 -- 172.21.15.106:0/925520813 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c006250 con 0x7f4bac15bd50 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.328+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bac161620 con 0x7f4bac15bd50 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.332+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bac069d40 con 0x7f4bac15bd50 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.332+0000 7f4b93fff640 1 -- 172.21.15.106:0/925520813 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b9c01c070 con 0x7f4bac15bd50 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.332+0000 7f4b93fff640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9806f430 0x7f4b98071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.332+0000 7f4baa7fc640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9806f430 0x7f4b98071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.332+0000 7f4b93fff640 1 -- 172.21.15.106:0/925520813 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4b9c08f7c0 con 0x7f4bac15bd50 2024-04-25T11:20:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.332+0000 7f4baa7fc640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9806f430 0x7f4b98071940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4bac1f9ae0 tx=0x7f4ba0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.336+0000 7f4b93fff640 1 -- 172.21.15.106:0/925520813 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4bac0b78e0 con 0x7f4bac15bd50 2024-04-25T11:20:08.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.496+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bac0b40c0 con 0x7f4bac15bd50 2024-04-25T11:20:08.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.496+0000 7f4b93fff640 1 -- 172.21.15.106:0/925520813 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16834 (secure 0 0 0) 0x7f4b9c05c630 con 0x7f4bac15bd50 2024-04-25T11:20:08.497 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:08.497 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:07.237284+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9806f430 msgr2=0x7f4b98071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9806f430 0x7f4b98071940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4bac1f9ae0 tx=0x7f4ba0009040 comp rx=0 tx=0).stop 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bac15bd50 msgr2=0x7f4bac0b6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bac15bd50 0x7f4bac0b6f90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c011600 tx=0x7f4b9c011ac0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 shutdown_connections 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4bac160b50 0x7f4bac0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bac0b74d0 0x7f4bac160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b9806f430 0x7f4b98071940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 --2- 172.21.15.106:0/925520813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bac15bd50 0x7f4bac0b6f90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 >> 172.21.15.106:0/925520813 conn(0x7f4bac0cffb0 msgr2=0x7f4bac0b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 shutdown_connections 2024-04-25T11:20:08.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.500+0000 7f4bb1904640 1 -- 172.21.15.106:0/925520813 wait complete. 2024-04-25T11:20:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:08.517 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:08.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.656+0000 7f22f8e36640 1 Processor -- start 2024-04-25T11:20:08.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.656+0000 7f22f8e36640 1 -- start start 2024-04-25T11:20:08.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f8e36640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 0x7f22f4072a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f8e36640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f22f4072f60 0x7f22f415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f8e36640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f4073370 con 0x7f22f40b6120 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f8e36640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f22f40b6530 con 0x7f22f4072f60 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f8e36640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f22f40b5000 con 0x7f22f40706c0 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f22f4072f60 0x7f22f415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f22f4072f60 0x7f22f415fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41860/0 (socket says 172.21.15.106:41860) 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f2d9d640 1 -- 172.21.15.106:0/3097829980 learned_addr learned my addr 172.21.15.106:0/3097829980 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f1d9b640 1 --2- 172.21.15.106:0/3097829980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 0x7f22f4072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f2d9d640 1 -- 172.21.15.106:0/3097829980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 msgr2=0x7f22f4072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f2d9d640 1 --2- 172.21.15.106:0/3097829980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 0x7f22f4072a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f2d9d640 1 -- 172.21.15.106:0/3097829980 >> v1:172.21.15.134:6789/0 conn(0x7f22f40b6120 legacy=0x7f22f415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:08.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.660+0000 7f22f2d9d640 1 -- 172.21.15.106:0/3097829980 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f40b43f0 con 0x7f22f4072f60 2024-04-25T11:20:08.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f2d9d640 1 --2- 172.21.15.106:0/3097829980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4072f60 0x7f22f415fc70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f22f40b5180 tx=0x7f22ec01dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6b97f6e5e01073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:08.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f159a640 1 -- 172.21.15.106:0/3097829980 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00a2d0 con 0x7f22f4072f60 2024-04-25T11:20:08.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f159a640 1 -- 172.21.15.106:0/3097829980 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec0a6920 con 0x7f22f4072f60 2024-04-25T11:20:08.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f159a640 1 -- 172.21.15.106:0/3097829980 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0a6cf0 con 0x7f22f4072f60 2024-04-25T11:20:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 -- 172.21.15.106:0/3097829980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4072f60 msgr2=0x7f22f415fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 --2- 172.21.15.106:0/3097829980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4072f60 0x7f22f415fc70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f22f40b5180 tx=0x7f22ec01dae0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 -- 172.21.15.106:0/3097829980 shutdown_connections 2024-04-25T11:20:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 --2- 172.21.15.106:0/3097829980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4072f60 0x7f22f415fc70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 --2- 172.21.15.106:0/3097829980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 0x7f22f4072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 -- 172.21.15.106:0/3097829980 >> 172.21.15.106:0/3097829980 conn(0x7f22f406c9f0 msgr2=0x7f22f406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 -- 172.21.15.106:0/3097829980 shutdown_connections 2024-04-25T11:20:08.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 -- 172.21.15.106:0/3097829980 wait complete. 2024-04-25T11:20:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 Processor -- start 2024-04-25T11:20:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.664+0000 7f22f8e36640 1 -- start start 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 0x7f22f415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22f4072f60 0x7f22f415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f40b6120 0x7f22f415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4070c50 con 0x7f22f40b6120 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4070ad0 con 0x7f22f4072f60 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f22f41601b0 con 0x7f22f40706c0 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f40b6120 0x7f22f415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f40b6120 0x7f22f415dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41876/0 (socket says 172.21.15.106:41876) 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 -- 172.21.15.106:0/2407140306 learned_addr learned my addr 172.21.15.106:0/2407140306 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f1d9b640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22f4072f60 0x7f22f415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f259c640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 0x7f22f415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 -- 172.21.15.106:0/2407140306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 msgr2=0x7f22f415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 0x7f22f415d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 -- 172.21.15.106:0/2407140306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22f4072f60 msgr2=0x7f22f415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22f4072f60 0x7f22f415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 -- 172.21.15.106:0/2407140306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f415c7f0 con 0x7f22f40b6120 2024-04-25T11:20:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f2d9d640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f40b6120 0x7f22f415dd10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22ec01c190 tx=0x7f22ec01a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22e2ffd640 1 -- 172.21.15.106:0/2407140306 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec008700 con 0x7f22f40b6120 2024-04-25T11:20:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22e2ffd640 1 -- 172.21.15.106:0/2407140306 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec0bd070 con 0x7f22f40b6120 2024-04-25T11:20:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f4155320 con 0x7f22f40b6120 2024-04-25T11:20:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22e2ffd640 1 -- 172.21.15.106:0/2407140306 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec008ae0 con 0x7f22f40b6120 2024-04-25T11:20:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f415cd70 con 0x7f22f40b6120 2024-04-25T11:20:08.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f4155070 con 0x7f22f40b6120 2024-04-25T11:20:08.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22e2ffd640 1 -- 172.21.15.106:0/2407140306 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22ec008c80 con 0x7f22f40b6120 2024-04-25T11:20:08.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22e2ffd640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22d406f430 0x7f22d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.668+0000 7f22f259c640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22d406f430 0x7f22d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.672+0000 7f22e2ffd640 1 -- 172.21.15.106:0/2407140306 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f22ec12a010 con 0x7f22f40b6120 2024-04-25T11:20:08.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.672+0000 7f22f259c640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22d406f430 0x7f22d4071940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22e4007fd0 tx=0x7f22e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:08.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.676+0000 7f22e2ffd640 1 -- 172.21.15.106:0/2407140306 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22ec0c6020 con 0x7f22f40b6120 2024-04-25T11:20:08.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.824+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f22f406fec0 con 0x7f22f40b6120 2024-04-25T11:20:08.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.824+0000 7f22e2ffd640 1 -- 172.21.15.106:0/2407140306 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f22ec017700 con 0x7f22f40b6120 2024-04-25T11:20:08.824 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:08.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22d406f430 msgr2=0x7f22d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22d406f430 0x7f22d4071940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22e4007fd0 tx=0x7f22e4009040 comp rx=0 tx=0).stop 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f40b6120 msgr2=0x7f22f415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f40b6120 0x7f22f415dd10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22ec01c190 tx=0x7f22ec01a880 comp rx=0 tx=0).stop 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 shutdown_connections 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f22d406f430 0x7f22d4071940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f40b6120 0x7f22f415dd10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22f4072f60 0x7f22f415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 --2- 172.21.15.106:0/2407140306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22f40706c0 0x7f22f415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 >> 172.21.15.106:0/2407140306 conn(0x7f22f406c9f0 msgr2=0x7f22f40b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 shutdown_connections 2024-04-25T11:20:08.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.828+0000 7f22f8e36640 1 -- 172.21.15.106:0/2407140306 wait complete. 2024-04-25T11:20:08.844 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:08.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb2998e7640 1 Processor -- start 2024-04-25T11:20:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb2998e7640 1 -- start start 2024-04-25T11:20:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb2998e7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2940cd930 0x7fb2940d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb2998e7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb2940d1c80 0x7fb2940b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb2998e7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2940cdd40 con 0x7fb294143fa0 2024-04-25T11:20:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb2998e7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb2940d2090 con 0x7fb2940d1c80 2024-04-25T11:20:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb2998e7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb294073300 con 0x7fb2940cd930 2024-04-25T11:20:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb2937fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb2940d1c80 0x7fb2940b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb292ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb294143fa0 0x7fb294072a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37260/0 (socket says 172.21.15.106:37260) 2024-04-25T11:20:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:08.996+0000 7fb292ffd640 1 -- 172.21.15.106:0/2946072720 learned_addr learned my addr 172.21.15.106:0/2946072720 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2927fc640 1 --2- 172.21.15.106:0/2946072720 >> v2:172.21.15.134:3300/0 conn(0x7fb2940cd930 0x7fb2940d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2937fe640 1 -- 172.21.15.106:0/2946072720 >> v2:172.21.15.134:3300/0 conn(0x7fb2940cd930 msgr2=0x7fb2940d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2937fe640 1 --2- 172.21.15.106:0/2946072720 >> v2:172.21.15.134:3300/0 conn(0x7fb2940cd930 0x7fb2940d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2937fe640 1 -- 172.21.15.106:0/2946072720 >> v1:172.21.15.106:6789/0 conn(0x7fb294143fa0 legacy=0x7fb294072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2937fe640 1 -- 172.21.15.106:0/2946072720 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2940735e0 con 0x7fb2940d1c80 2024-04-25T11:20:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2937fe640 1 --2- 172.21.15.106:0/2946072720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb2940b5840 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb2940721f0 tx=0x7fb28c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90120b7ee11baf09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb291ffb640 1 -- 172.21.15.106:0/2946072720 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0b2070 con 0x7fb2940d1c80 2024-04-25T11:20:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb291ffb640 1 -- 172.21.15.106:0/2946072720 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c0a58e0 con 0x7fb2940d1c80 2024-04-25T11:20:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb291ffb640 1 -- 172.21.15.106:0/2946072720 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0a5cb0 con 0x7fb2940d1c80 2024-04-25T11:20:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2998e7640 1 -- 172.21.15.106:0/2946072720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 msgr2=0x7fb2940b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2998e7640 1 --2- 172.21.15.106:0/2946072720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb2940b5840 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb2940721f0 tx=0x7fb28c01c930 comp rx=0 tx=0).stop 2024-04-25T11:20:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2998e7640 1 -- 172.21.15.106:0/2946072720 shutdown_connections 2024-04-25T11:20:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2998e7640 1 --2- 172.21.15.106:0/2946072720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb2940b5840 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2998e7640 1 --2- 172.21.15.106:0/2946072720 >> v2:172.21.15.134:3300/0 conn(0x7fb2940cd930 0x7fb2940d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2998e7640 1 -- 172.21.15.106:0/2946072720 >> 172.21.15.106:0/2946072720 conn(0x7fb29406c9f0 msgr2=0x7fb29406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2998e7640 1 -- 172.21.15.106:0/2946072720 shutdown_connections 2024-04-25T11:20:09.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.000+0000 7fb2998e7640 1 -- 172.21.15.106:0/2946072720 wait complete. 2024-04-25T11:20:09.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 Processor -- start 2024-04-25T11:20:09.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 -- start start 2024-04-25T11:20:09.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2940cd930 0x7fb294070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb29413fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb294143fa0 0x7fb294159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb29416ab10 con 0x7fb2940d1c80 2024-04-25T11:20:09.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb29416a990 con 0x7fb2940cd930 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb29416ac90 con 0x7fb294143fa0 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb29413fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb29413fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41890/0 (socket says 172.21.15.106:41890) 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 -- 172.21.15.106:0/535893146 learned_addr learned my addr 172.21.15.106:0/535893146 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2937fe640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb294143fa0 0x7fb294159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb292ffd640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2940cd930 0x7fb294070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 -- 172.21.15.106:0/535893146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb294143fa0 msgr2=0x7fb294159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb294143fa0 0x7fb294159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 -- 172.21.15.106:0/535893146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2940cd930 msgr2=0x7fb294070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2940cd930 0x7fb294070610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 -- 172.21.15.106:0/535893146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29406ea20 con 0x7fb2940d1c80 2024-04-25T11:20:09.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2927fc640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb29413fb80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb28800d620 tx=0x7fb28800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb27bfff640 1 -- 172.21.15.106:0/535893146 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28800b620 con 0x7fb2940d1c80 2024-04-25T11:20:09.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb27bfff640 1 -- 172.21.15.106:0/535893146 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb288019070 con 0x7fb2940d1c80 2024-04-25T11:20:09.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29406ecc0 con 0x7fb2940d1c80 2024-04-25T11:20:09.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb27bfff640 1 -- 172.21.15.106:0/535893146 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28800b9e0 con 0x7fb2940d1c80 2024-04-25T11:20:09.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.004+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29406f280 con 0x7fb2940d1c80 2024-04-25T11:20:09.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.008+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb294050750 con 0x7fb2940d1c80 2024-04-25T11:20:09.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.008+0000 7fb27bfff640 1 -- 172.21.15.106:0/535893146 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb28800bb80 con 0x7fb2940d1c80 2024-04-25T11:20:09.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.008+0000 7fb27bfff640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb28006f430 0x7fb280071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.008+0000 7fb27bfff640 1 -- 172.21.15.106:0/535893146 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb288090250 con 0x7fb2940d1c80 2024-04-25T11:20:09.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.008+0000 7fb292ffd640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb28006f430 0x7fb280071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.008+0000 7fb292ffd640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb28006f430 0x7fb280071940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb284000f70 tx=0x7fb284009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.012+0000 7fb27bfff640 1 -- 172.21.15.106:0/535893146 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb28805d130 con 0x7fb2940d1c80 2024-04-25T11:20:09.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.160+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2940b4e20 con 0x7fb2940d1c80 2024-04-25T11:20:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.160+0000 7fb27bfff640 1 -- 172.21.15.106:0/535893146 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fb28805ca30 con 0x7fb2940d1c80 2024-04-25T11:20:09.161 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb28006f430 msgr2=0x7fb280071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb28006f430 0x7fb280071940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb284000f70 tx=0x7fb284009040 comp rx=0 tx=0).stop 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 msgr2=0x7fb29413fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb29413fb80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb28800d620 tx=0x7fb28800dae0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 shutdown_connections 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb294143fa0 0x7fb294159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb28006f430 0x7fb280071940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2940d1c80 0x7fb29413fb80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 --2- 172.21.15.106:0/535893146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2940cd930 0x7fb294070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 >> 172.21.15.106:0/535893146 conn(0x7fb29406c9f0 msgr2=0x7fb2941464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 shutdown_connections 2024-04-25T11:20:09.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.164+0000 7fb2998e7640 1 -- 172.21.15.106:0/535893146 wait complete. 2024-04-25T11:20:09.181 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:09.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb8559640 1 Processor -- start 2024-04-25T11:20:09.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb8559640 1 -- start start 2024-04-25T11:20:09.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb8559640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8fb01669f0 0x7f8fb0168e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb8559640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb00d0cd0 con 0x7f8fb0143f10 2024-04-25T11:20:09.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb8559640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb00d0fa0 con 0x7f8fb00d1740 2024-04-25T11:20:09.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb8559640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb0144320 con 0x7f8fb01669f0 2024-04-25T11:20:09.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb62f5640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8fb0143f10 0x7f8fb00d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37274/0 (socket says 172.21.15.106:37274) 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb62f5640 1 -- 172.21.15.106:0/1760905295 learned_addr learned my addr 172.21.15.106:0/1760905295 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb6af6640 1 --2- 172.21.15.106:0/1760905295 >> v2:172.21.15.134:3300/0 conn(0x7f8fb01669f0 0x7f8fb0168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880347485 0 0) 0x7f8fb00d0cd0 con 0x7f8fb0143f10 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f9c003530 con 0x7f8fb0143f10 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4137323904 0 0) 0x7f8f9c003530 con 0x7f8fb0143f10 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 >> v2:172.21.15.134:3300/0 conn(0x7f8fb01669f0 msgr2=0x7f8fb0168e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 --2- 172.21.15.106:0/1760905295 >> v2:172.21.15.134:3300/0 conn(0x7f8fb01669f0 0x7f8fb0168e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 >> v1:172.21.15.134:6789/0 conn(0x7f8fb00d1740 legacy=0x7f8fb00d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb01618c0 con 0x7f8fb0143f10 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f8fa8001680 con 0x7f8fb0143f10 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 >> v1:172.21.15.106:6789/0 conn(0x7f8fb0143f10 legacy=0x7f8fb00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9c0039c0 0x7f8f9c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f9c004810 0x7f8f9c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8f9c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c00c260 con 0x7f8f9c005360 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c00c410 con 0x7f8f9c004810 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c00c610 con 0x7f8f9c0039c0 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.336+0000 7f8fb6af6640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8f9c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb5af4640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f9c004810 0x7f8f9c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb62f5640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9c0039c0 0x7f8f9c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb6af6640 1 -- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9c0039c0 msgr2=0x7f8f9c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb6af6640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9c0039c0 0x7f8f9c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb6af6640 1 -- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f9c004810 msgr2=0x7f8f9c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:09.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb6af6640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f9c004810 0x7f8f9c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb6af6640 1 -- 172.21.15.106:0/1760905295 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c003530 con 0x7f8f9c005360 2024-04-25T11:20:09.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb62f5640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9c0039c0 0x7f8f9c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:09.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb6af6640 1 -- 172.21.15.106:0/1760905295 reap_dead start 2024-04-25T11:20:09.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb6af6640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8f9c00b9d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00d2310 tx=0x7f8fac01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bad738ec246f8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac009c10 con 0x7f8f9c005360 2024-04-25T11:20:09.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac003ba0 con 0x7f8f9c005360 2024-04-25T11:20:09.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb52f3640 1 -- 172.21.15.106:0/1760905295 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0a5740 con 0x7f8f9c005360 2024-04-25T11:20:09.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb8559640 1 -- 172.21.15.106:0/1760905295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 msgr2=0x7f8f9c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb8559640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8f9c00b9d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00d2310 tx=0x7f8fac01cc60 comp rx=0 tx=0).stop 2024-04-25T11:20:09.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb8559640 1 -- 172.21.15.106:0/1760905295 shutdown_connections 2024-04-25T11:20:09.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb8559640 1 --2- 172.21.15.106:0/1760905295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8f9c00b9d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb8559640 1 -- 172.21.15.106:0/1760905295 >> 172.21.15.106:0/1760905295 conn(0x7f8fb006c9f0 msgr2=0x7f8fb006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb8559640 1 -- 172.21.15.106:0/1760905295 shutdown_connections 2024-04-25T11:20:09.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.340+0000 7f8fb8559640 1 -- 172.21.15.106:0/1760905295 wait complete. 2024-04-25T11:20:09.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 Processor -- start 2024-04-25T11:20:09.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 -- start start 2024-04-25T11:20:09.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8fb015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb0135b00 0x7f8fb0144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fb0145360 0x7f8fb0167e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb00d05c0 con 0x7f8f9c005360 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01625b0 con 0x7f8fb0135b00 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb00d0740 con 0x7f8fb0145360 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8fb015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8fb015c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41908/0 (socket says 172.21.15.106:41908) 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 -- 172.21.15.106:0/3208722356 learned_addr learned my addr 172.21.15.106:0/3208722356 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb5af4640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb0135b00 0x7f8fb0144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb6af6640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fb0145360 0x7f8fb0167e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 -- 172.21.15.106:0/3208722356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fb0145360 msgr2=0x7f8fb0167e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fb0145360 0x7f8fb0167e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 -- 172.21.15.106:0/3208722356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb0135b00 msgr2=0x7f8fb0144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb0135b00 0x7f8fb0144e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 -- 172.21.15.106:0/3208722356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb0167050 con 0x7f8f9c005360 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb62f5640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8fb015c3c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8001210 tx=0x7f8fa800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fa6ffd640 1 -- 172.21.15.106:0/3208722356 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8019070 con 0x7f8f9c005360 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fa6ffd640 1 -- 172.21.15.106:0/3208722356 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa800de50 con 0x7f8f9c005360 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb0167330 con 0x7f8f9c005360 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fa6ffd640 1 -- 172.21.15.106:0/3208722356 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80047f0 con 0x7f8f9c005360 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.344+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01678f0 con 0x7f8f9c005360 2024-04-25T11:20:09.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.348+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb0050750 con 0x7f8f9c005360 2024-04-25T11:20:09.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.348+0000 7f8fa6ffd640 1 -- 172.21.15.106:0/3208722356 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8fa801c070 con 0x7f8f9c005360 2024-04-25T11:20:09.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.348+0000 7f8fa6ffd640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8f9406f430 0x7f8f94071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.348+0000 7f8fa6ffd640 1 -- 172.21.15.106:0/3208722356 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8fa808efc0 con 0x7f8f9c005360 2024-04-25T11:20:09.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.348+0000 7f8fb5af4640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8f9406f430 0x7f8f94071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.348+0000 7f8fb5af4640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8f9406f430 0x7f8f94071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fb01467f0 tx=0x7f8fa0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.352+0000 7f8fa6ffd640 1 -- 172.21.15.106:0/3208722356 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8fa8023020 con 0x7f8f9c005360 2024-04-25T11:20:09.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.508+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8fb00b4db0 con 0x7f8f9c005360 2024-04-25T11:20:09.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fa6ffd640 1 -- 172.21.15.106:0/3208722356 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f8fa805bfa0 con 0x7f8f9c005360 2024-04-25T11:20:09.510 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:09.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8f9406f430 msgr2=0x7f8f94071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8f9406f430 0x7f8f94071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fb01467f0 tx=0x7f8fa0009040 comp rx=0 tx=0).stop 2024-04-25T11:20:09.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 msgr2=0x7f8fb015c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8fb015c3c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8001210 tx=0x7f8fa800dc00 comp rx=0 tx=0).stop 2024-04-25T11:20:09.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 shutdown_connections 2024-04-25T11:20:09.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fb0145360 0x7f8fb0167e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb0135b00 0x7f8fb0144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8f9406f430 0x7f8f94071940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 --2- 172.21.15.106:0/3208722356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f9c005360 0x7f8fb015c3c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 >> 172.21.15.106:0/3208722356 conn(0x7f8fb006c9f0 msgr2=0x7f8fb006f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 shutdown_connections 2024-04-25T11:20:09.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.512+0000 7f8fb8559640 1 -- 172.21.15.106:0/3208722356 wait complete. 2024-04-25T11:20:09.533 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b90a37640 1 Processor -- start 2024-04-25T11:20:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b90a37640 1 -- start start 2024-04-25T11:20:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b90a37640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c073460 0x7f4b8c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b90a37640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b8c15e6d0 0x7f4b8c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b90a37640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4b8c06cd60 0x7f4b8c06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b90a37640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8c15f1a0 con 0x7f4b8c15e6d0 2024-04-25T11:20:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b90a37640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8c072fd0 con 0x7f4b8c06cd60 2024-04-25T11:20:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b90a37640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c06d170 con 0x7f4b8c073460 2024-04-25T11:20:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b89d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b8c15e6d0 0x7f4b8c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b89d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b8c15e6d0 0x7f4b8c06c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41922/0 (socket says 172.21.15.106:41922) 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b89d9b640 1 -- 172.21.15.106:0/2402174682 learned_addr learned my addr 172.21.15.106:0/2402174682 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b89d9b640 1 -- 172.21.15.106:0/2402174682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c073460 msgr2=0x7f4b8c15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b8a59c640 1 --2- 172.21.15.106:0/2402174682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c073460 0x7f4b8c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b8ad9d640 1 --2- 172.21.15.106:0/2402174682 >> v2:172.21.15.134:3300/0 conn(0x7f4b8c06cd60 0x7f4b8c06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b89d9b640 1 --2- 172.21.15.106:0/2402174682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c073460 0x7f4b8c15e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b89d9b640 1 -- 172.21.15.106:0/2402174682 >> v2:172.21.15.134:3300/0 conn(0x7f4b8c06cd60 msgr2=0x7f4b8c06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b89d9b640 1 --2- 172.21.15.106:0/2402174682 >> v2:172.21.15.134:3300/0 conn(0x7f4b8c06cd60 0x7f4b8c06f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.680+0000 7f4b89d9b640 1 -- 172.21.15.106:0/2402174682 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c144350 con 0x7f4b8c15e6d0 2024-04-25T11:20:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b89d9b640 1 --2- 172.21.15.106:0/2402174682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c15e6d0 0x7f4b8c06c820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c06d530 tx=0x7f4b8001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9089f9d665a1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b8959a640 1 -- 172.21.15.106:0/2402174682 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b800aa610 con 0x7f4b8c15e6d0 2024-04-25T11:20:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b8959a640 1 -- 172.21.15.106:0/2402174682 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b800af070 con 0x7f4b8c15e6d0 2024-04-25T11:20:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b8959a640 1 -- 172.21.15.106:0/2402174682 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b800aa9d0 con 0x7f4b8c15e6d0 2024-04-25T11:20:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 -- 172.21.15.106:0/2402174682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c15e6d0 msgr2=0x7f4b8c06c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2402174682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c15e6d0 0x7f4b8c06c820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c06d530 tx=0x7f4b8001c4f0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 -- 172.21.15.106:0/2402174682 shutdown_connections 2024-04-25T11:20:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2402174682 >> v2:172.21.15.134:3300/0 conn(0x7f4b8c06cd60 0x7f4b8c06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2402174682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c15e6d0 0x7f4b8c06c820 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2402174682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c073460 0x7f4b8c15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 -- 172.21.15.106:0/2402174682 >> 172.21.15.106:0/2402174682 conn(0x7f4b8c0b3560 msgr2=0x7f4b8c0b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 -- 172.21.15.106:0/2402174682 shutdown_connections 2024-04-25T11:20:09.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.684+0000 7f4b90a37640 1 -- 172.21.15.106:0/2402174682 wait complete. 2024-04-25T11:20:09.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 Processor -- start 2024-04-25T11:20:09.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 -- start start 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c06cd60 0x7f4b8c14f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b8c073460 0x7f4b8c13e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c15e6d0 0x7f4b8c15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c0739f0 con 0x7f4b8c06cd60 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c073870 con 0x7f4b8c073460 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c070130 con 0x7f4b8c15e6d0 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c06cd60 0x7f4b8c14f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c06cd60 0x7f4b8c14f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41926/0 (socket says 172.21.15.106:41926) 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 -- 172.21.15.106:0/2731865994 learned_addr learned my addr 172.21.15.106:0/2731865994 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 -- 172.21.15.106:0/2731865994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c15e6d0 msgr2=0x7f4b8c15f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c15e6d0 0x7f4b8c15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 -- 172.21.15.106:0/2731865994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b8c073460 msgr2=0x7f4b8c13e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b8c073460 0x7f4b8c13e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 -- 172.21.15.106:0/2731865994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c161170 con 0x7f4b8c06cd60 2024-04-25T11:20:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b8a59c640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c06cd60 0x7f4b8c14f1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c7f0 tx=0x7f4b7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b7affd640 1 -- 172.21.15.106:0/2731865994 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c004230 con 0x7f4b8c06cd60 2024-04-25T11:20:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b7affd640 1 -- 172.21.15.106:0/2731865994 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7c0043d0 con 0x7f4b8c06cd60 2024-04-25T11:20:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c15db00 con 0x7f4b8c06cd60 2024-04-25T11:20:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b7affd640 1 -- 172.21.15.106:0/2731865994 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c003860 con 0x7f4b8c06cd60 2024-04-25T11:20:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c13d570 con 0x7f4b8c06cd60 2024-04-25T11:20:09.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8c163e50 con 0x7f4b8c06cd60 2024-04-25T11:20:09.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.688+0000 7f4b7affd640 1 -- 172.21.15.106:0/2731865994 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b7c003c10 con 0x7f4b8c06cd60 2024-04-25T11:20:09.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.692+0000 7f4b7affd640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b6c06f400 0x7f4b6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.692+0000 7f4b89d9b640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b6c06f400 0x7f4b6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.692+0000 7f4b7affd640 1 -- 172.21.15.106:0/2731865994 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4b7c08f3e0 con 0x7f4b8c06cd60 2024-04-25T11:20:09.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.692+0000 7f4b89d9b640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b6c06f400 0x7f4b6c071910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b800ae000 tx=0x7f4b80004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.696+0000 7f4b7affd640 1 -- 172.21.15.106:0/2731865994 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b8c15db00 con 0x7f4b8c06cd60 2024-04-25T11:20:09.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.852+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4b8c0708c0 con 0x7f4b8c06cd60 2024-04-25T11:20:09.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b7affd640 1 -- 172.21.15.106:0/2731865994 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f4b7c015020 con 0x7f4b8c06cd60 2024-04-25T11:20:09.854 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b6c06f400 msgr2=0x7f4b6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b6c06f400 0x7f4b6c071910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b800ae000 tx=0x7f4b80004740 comp rx=0 tx=0).stop 2024-04-25T11:20:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c06cd60 msgr2=0x7f4b8c14f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c06cd60 0x7f4b8c14f1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c7f0 tx=0x7f4b7c00ccb0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 shutdown_connections 2024-04-25T11:20:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b8c15e6d0 0x7f4b8c15f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b8c073460 0x7f4b8c13e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f4b6c06f400 0x7f4b6c071910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 --2- 172.21.15.106:0/2731865994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b8c06cd60 0x7f4b8c14f1f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 >> 172.21.15.106:0/2731865994 conn(0x7f4b8c0b3560 msgr2=0x7f4b8c145660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 shutdown_connections 2024-04-25T11:20:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:09.856+0000 7f4b90a37640 1 -- 172.21.15.106:0/2731865994 wait complete. 2024-04-25T11:20:09.875 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi134.front.sepia.ceph.com... 2024-04-25T11:20:09.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:09.875 DEBUG:teuthology.orchestra.run.smithi134:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T11:20:09.988 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:09.988 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:20:09.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:09.988 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:20:10.006 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:10.006 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:10.016 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:10.016 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:10.063 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:10.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136f1f9640 1 Processor -- start 2024-04-25T11:20:10.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136f1f9640 1 -- start start 2024-04-25T11:20:10.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136f1f9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f13680ba970 0x7f13680d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136f1f9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13680d2160 0x7f13680d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136f1f9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13680d2ab0 con 0x7f13680b5120 2024-04-25T11:20:10.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136f1f9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f13680d1100 con 0x7f13680d2160 2024-04-25T11:20:10.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136f1f9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f13680d0530 con 0x7f13680ba970 2024-04-25T11:20:10.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f1367fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13680d2160 0x7f13680d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f1367fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13680d2160 0x7f13680d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41938/0 (socket says 172.21.15.106:41938) 2024-04-25T11:20:10.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f1367fff640 1 -- 172.21.15.106:0/2363484261 learned_addr learned my addr 172.21.15.106:0/2363484261 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:10.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136cf95640 1 --2- 172.21.15.106:0/2363484261 >> v2:172.21.15.134:3300/0 conn(0x7f13680ba970 0x7f13680d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f1367fff640 1 -- 172.21.15.106:0/2363484261 >> v2:172.21.15.134:3300/0 conn(0x7f13680ba970 msgr2=0x7f13680d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f1367fff640 1 --2- 172.21.15.106:0/2363484261 >> v2:172.21.15.134:3300/0 conn(0x7f13680ba970 0x7f13680d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f1367fff640 1 -- 172.21.15.106:0/2363484261 >> v1:172.21.15.134:6789/0 conn(0x7f13680b5120 legacy=0x7f13680b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f1367fff640 1 -- 172.21.15.106:0/2363484261 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13680b5c40 con 0x7f13680d2160 2024-04-25T11:20:10.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.256+0000 7f136cf95640 1 --2- 172.21.15.106:0/2363484261 >> v2:172.21.15.134:3300/0 conn(0x7f13680ba970 0x7f13680d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:10.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f1367fff640 1 --2- 172.21.15.106:0/2363484261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680d2160 0x7f13680d2570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f136808e700 tx=0x7f135c01c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a982688cbb50499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f13677fe640 1 -- 172.21.15.106:0/2363484261 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0b0070 con 0x7f13680d2160 2024-04-25T11:20:10.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f13677fe640 1 -- 172.21.15.106:0/2363484261 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c0ab600 con 0x7f13680d2160 2024-04-25T11:20:10.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f13677fe640 1 -- 172.21.15.106:0/2363484261 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0ab9d0 con 0x7f13680d2160 2024-04-25T11:20:10.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f136f1f9640 1 -- 172.21.15.106:0/2363484261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680d2160 msgr2=0x7f13680d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2363484261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680d2160 0x7f13680d2570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f136808e700 tx=0x7f135c01c300 comp rx=0 tx=0).stop 2024-04-25T11:20:10.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f136f1f9640 1 -- 172.21.15.106:0/2363484261 shutdown_connections 2024-04-25T11:20:10.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2363484261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680d2160 0x7f13680d2570 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2363484261 >> v2:172.21.15.134:3300/0 conn(0x7f13680ba970 0x7f13680d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f136f1f9640 1 -- 172.21.15.106:0/2363484261 >> 172.21.15.106:0/2363484261 conn(0x7f13680d57e0 msgr2=0x7f13680d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:10.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f136f1f9640 1 -- 172.21.15.106:0/2363484261 shutdown_connections 2024-04-25T11:20:10.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.260+0000 7f136f1f9640 1 -- 172.21.15.106:0/2363484261 wait complete. 2024-04-25T11:20:10.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.264+0000 7f136f1f9640 1 Processor -- start 2024-04-25T11:20:10.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.264+0000 7f136f1f9640 1 -- start start 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13680b5120 0x7f136815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680ba970 0x7f136809f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13680d2160 0x7f136809fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13680d1ad0 con 0x7f13680ba970 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13681605b0 con 0x7f13680b5120 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13680b4da0 con 0x7f13680d2160 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680ba970 0x7f136809f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680ba970 0x7f136809f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41952/0 (socket says 172.21.15.106:41952) 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 -- 172.21.15.106:0/2021578832 learned_addr learned my addr 172.21.15.106:0/2021578832 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136cf95640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13680b5120 0x7f136815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136d796640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13680d2160 0x7f136809fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 -- 172.21.15.106:0/2021578832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13680d2160 msgr2=0x7f136809fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13680d2160 0x7f136809fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 -- 172.21.15.106:0/2021578832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13680b5120 msgr2=0x7f136815c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13680b5120 0x7f136815c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 -- 172.21.15.106:0/2021578832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13681fd930 con 0x7f13680ba970 2024-04-25T11:20:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f1367fff640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680ba970 0x7f136809f520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f135c019f30 tx=0x7f135c0190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f13657fa640 1 -- 172.21.15.106:0/2021578832 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0b0040 con 0x7f13680ba970 2024-04-25T11:20:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f13657fa640 1 -- 172.21.15.106:0/2021578832 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c0bc070 con 0x7f13680ba970 2024-04-25T11:20:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13681fdc10 con 0x7f13680ba970 2024-04-25T11:20:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f13657fa640 1 -- 172.21.15.106:0/2021578832 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0abc90 con 0x7f13680ba970 2024-04-25T11:20:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1368161f20 con 0x7f13680ba970 2024-04-25T11:20:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.268+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13681fd6a0 con 0x7f13680ba970 2024-04-25T11:20:10.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.272+0000 7f13657fa640 1 -- 172.21.15.106:0/2021578832 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f135c003620 con 0x7f13680ba970 2024-04-25T11:20:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.272+0000 7f13657fa640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f135406f430 0x7f1354071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.272+0000 7f13657fa640 1 -- 172.21.15.106:0/2021578832 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f135c12a100 con 0x7f13680ba970 2024-04-25T11:20:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.272+0000 7f136cf95640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f135406f430 0x7f1354071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.272+0000 7f136cf95640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f135406f430 0x7f1354071940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13680b8320 tx=0x7f1358008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.276+0000 7f13657fa640 1 -- 172.21.15.106:0/2021578832 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13680d2ab0 con 0x7f13680ba970 2024-04-25T11:20:10.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.436+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13680a0cf0 con 0x7f13680ba970 2024-04-25T11:20:10.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f13657fa640 1 -- 172.21.15.106:0/2021578832 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16834 (secure 0 0 0) 0x7f135c0f7070 con 0x7f13680ba970 2024-04-25T11:20:10.439 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:07.237284+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:10.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f135406f430 msgr2=0x7f1354071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f135406f430 0x7f1354071940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13680b8320 tx=0x7f1358008040 comp rx=0 tx=0).stop 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680ba970 msgr2=0x7f136809f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680ba970 0x7f136809f520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f135c019f30 tx=0x7f135c0190a0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 shutdown_connections 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13680d2160 0x7f136809fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f135406f430 0x7f1354071940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13680ba970 0x7f136809f520 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 --2- 172.21.15.106:0/2021578832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13680b5120 0x7f136815c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 >> 172.21.15.106:0/2021578832 conn(0x7f13680d57e0 msgr2=0x7f13680b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 shutdown_connections 2024-04-25T11:20:10.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.440+0000 7f136f1f9640 1 -- 172.21.15.106:0/2021578832 wait complete. 2024-04-25T11:20:10.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:10.461 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:10.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff38909e640 1 Processor -- start 2024-04-25T11:20:10.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff38909e640 1 -- start start 2024-04-25T11:20:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff38909e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3840b6410 0x7ff3840b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff38909e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3840b8640 con 0x7ff3840d0b60 2024-04-25T11:20:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff38909e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3840b8d70 con 0x7ff3840b6d60 2024-04-25T11:20:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff38909e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff38406e510 con 0x7ff3840b6410 2024-04-25T11:20:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff382d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff3840d0b60 0x7ff38406d8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37276/0 (socket says 172.21.15.106:37276) 2024-04-25T11:20:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff382d9d640 1 -- 172.21.15.106:0/1738453102 learned_addr learned my addr 172.21.15.106:0/1738453102 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff38259c640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3840b6410 0x7ff3840b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596403061 0 0) 0x7ff3840b8640 con 0x7ff3840d0b60 2024-04-25T11:20:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff36c003530 con 0x7ff3840d0b60 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4255275954 0 0) 0x7ff36c003530 con 0x7ff3840d0b60 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3840b6410 msgr2=0x7ff3840b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3840b6410 0x7ff3840b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 >> v1:172.21.15.134:6789/0 conn(0x7ff3840b6d60 legacy=0x7ff38416a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38406e9a0 con 0x7ff3840d0b60 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7ff374001680 con 0x7ff3840d0b60 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 >> v1:172.21.15.106:6789/0 conn(0x7ff3840d0b60 legacy=0x7ff38406d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff36c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3840b6410 0x7ff36c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff36c004d50 0x7ff36c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff38406e510 con 0x7ff36c0039f0 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c00bb90 con 0x7ff36c004d50 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff36c00bd90 con 0x7ff3840b6410 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.616+0000 7ff382d9d640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff36c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38259c640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3840b6410 0x7ff36c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff382d9d640 1 -- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3840b6410 msgr2=0x7ff36c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff382d9d640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3840b6410 0x7ff36c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38359e640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff36c004d50 0x7ff36c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff382d9d640 1 -- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff36c004d50 msgr2=0x7ff36c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff382d9d640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff36c004d50 0x7ff36c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff382d9d640 1 -- 172.21.15.106:0/1738453102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36c003530 con 0x7ff36c0039f0 2024-04-25T11:20:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38359e640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff36c004d50 0x7ff36c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:10.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff382d9d640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff36c003e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff37400c210 tx=0x7ff37401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cf96f9d6454e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374007d40 con 0x7ff36c0039f0 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3740a9070 con 0x7ff36c0039f0 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff381d9b640 1 -- 172.21.15.106:0/1738453102 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374005d40 con 0x7ff36c0039f0 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38909e640 1 -- 172.21.15.106:0/1738453102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 msgr2=0x7ff36c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38909e640 1 --2- 172.21.15.106:0/1738453102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff36c003e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff37400c210 tx=0x7ff37401cca0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38359e640 1 -- 172.21.15.106:0/1738453102 reap_dead start 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38909e640 1 -- 172.21.15.106:0/1738453102 shutdown_connections 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38909e640 1 -- 172.21.15.106:0/1738453102 >> 172.21.15.106:0/1738453102 conn(0x7ff3840d0040 msgr2=0x7ff3840d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38909e640 1 -- 172.21.15.106:0/1738453102 shutdown_connections 2024-04-25T11:20:10.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.620+0000 7ff38909e640 1 -- 172.21.15.106:0/1738453102 wait complete. 2024-04-25T11:20:10.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 Processor -- start 2024-04-25T11:20:10.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 -- start start 2024-04-25T11:20:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff38416a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3841f9950 0x7ff38409db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3840cf270 0x7ff3840cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff38416c3f0 con 0x7ff36c0039f0 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff38406e1f0 con 0x7ff3840cf270 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff38416c570 con 0x7ff3841f9950 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff38416a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff38416a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41970/0 (socket says 172.21.15.106:41970) 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 -- 172.21.15.106:0/1030015526 learned_addr learned my addr 172.21.15.106:0/1030015526 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38359e640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3840cf270 0x7ff3840cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38259c640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3841f9950 0x7ff38409db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 -- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3841f9950 msgr2=0x7ff38409db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3841f9950 0x7ff38409db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 -- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3840cf270 msgr2=0x7ff3840cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3840cf270 0x7ff3840cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 -- 172.21.15.106:0/1030015526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38416abb0 con 0x7ff36c0039f0 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38359e640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3840cf270 0x7ff3840cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff382d9d640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff38416a670 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff37401cad0 tx=0x7ff36c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff36b7fe640 1 -- 172.21.15.106:0/1030015526 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374005d20 con 0x7ff36c0039f0 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff36b7fe640 1 -- 172.21.15.106:0/1030015526 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3740047f0 con 0x7ff36c0039f0 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38409e9b0 con 0x7ff36c0039f0 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff36b7fe640 1 -- 172.21.15.106:0/1030015526 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004ba0 con 0x7ff36c0039f0 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.624+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38409ef70 con 0x7ff36c0039f0 2024-04-25T11:20:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.628+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff384050750 con 0x7ff36c0039f0 2024-04-25T11:20:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.628+0000 7ff36b7fe640 1 -- 172.21.15.106:0/1030015526 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff3740a5780 con 0x7ff36c0039f0 2024-04-25T11:20:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.628+0000 7ff36b7fe640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff37006f400 0x7ff370071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.628+0000 7ff38259c640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff37006f400 0x7ff370071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.628+0000 7ff36b7fe640 1 -- 172.21.15.106:0/1030015526 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff3741292d0 con 0x7ff36c0039f0 2024-04-25T11:20:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.628+0000 7ff38259c640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff37006f400 0x7ff370071910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff378009160 tx=0x7ff378008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.632+0000 7ff36b7fe640 1 -- 172.21.15.106:0/1030015526 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff3740169b0 con 0x7ff36c0039f0 2024-04-25T11:20:10.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.780+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3840d2ff0 con 0x7ff36c0039f0 2024-04-25T11:20:10.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff36b7fe640 1 -- 172.21.15.106:0/1030015526 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7ff3740f6220 con 0x7ff36c0039f0 2024-04-25T11:20:10.782 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff37006f400 msgr2=0x7ff370071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff37006f400 0x7ff370071910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff378009160 tx=0x7ff378008040 comp rx=0 tx=0).stop 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 msgr2=0x7ff38416a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff38416a670 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff37401cad0 tx=0x7ff36c005840 comp rx=0 tx=0).stop 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 shutdown_connections 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3840cf270 0x7ff3840cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3841f9950 0x7ff38409db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff37006f400 0x7ff370071910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 --2- 172.21.15.106:0/1030015526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff36c0039f0 0x7ff38416a670 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 >> 172.21.15.106:0/1030015526 conn(0x7ff3840d0040 msgr2=0x7ff3840715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:10.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 shutdown_connections 2024-04-25T11:20:10.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.784+0000 7ff38909e640 1 -- 172.21.15.106:0/1030015526 wait complete. 2024-04-25T11:20:10.803 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:10.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a3e660640 1 Processor -- start 2024-04-25T11:20:10.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a3e660640 1 -- start start 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a3e660640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a380d0cf0 0x7f6a3806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a3e660640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6a380d1740 0x7f6a380d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a3e660640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a380d2090 con 0x7f6a3815d230 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a3e660640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6a380b68e0 con 0x7f6a380d1740 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a3e660640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6a38161210 con 0x7f6a380d0cf0 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a377fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6a380d1740 0x7f6a380d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a377fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6a380d1740 0x7f6a380d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41982/0 (socket says 172.21.15.106:41982) 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.948+0000 7f6a377fe640 1 -- 172.21.15.106:0/1817006545 learned_addr learned my addr 172.21.15.106:0/1817006545 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a37fff640 1 --2- 172.21.15.106:0/1817006545 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a380d0cf0 0x7f6a3806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a377fe640 1 -- 172.21.15.106:0/1817006545 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a380d0cf0 msgr2=0x7f6a3806ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a377fe640 1 --2- 172.21.15.106:0/1817006545 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a380d0cf0 0x7f6a3806ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a377fe640 1 -- 172.21.15.106:0/1817006545 >> v1:172.21.15.134:6789/0 conn(0x7f6a3815d230 legacy=0x7f6a3815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a377fe640 1 -- 172.21.15.106:0/1817006545 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a380b6290 con 0x7f6a380d1740 2024-04-25T11:20:10.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a37fff640 1 --2- 172.21.15.106:0/1817006545 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a380d0cf0 0x7f6a3806ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:10.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a377fe640 1 --2- 172.21.15.106:0/1817006545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d1740 0x7f6a380d1b50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00bb10 tx=0x7f6a2c01eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856db90e376bfc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a36ffd640 1 -- 172.21.15.106:0/1817006545 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0b4070 con 0x7f6a380d1740 2024-04-25T11:20:10.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a36ffd640 1 -- 172.21.15.106:0/1817006545 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0a78e0 con 0x7f6a380d1740 2024-04-25T11:20:10.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a36ffd640 1 -- 172.21.15.106:0/1817006545 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0a7cb0 con 0x7f6a380d1740 2024-04-25T11:20:10.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a3e660640 1 -- 172.21.15.106:0/1817006545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d1740 msgr2=0x7f6a380d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a3e660640 1 --2- 172.21.15.106:0/1817006545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d1740 0x7f6a380d1b50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00bb10 tx=0x7f6a2c01eb10 comp rx=0 tx=0).stop 2024-04-25T11:20:10.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a3e660640 1 -- 172.21.15.106:0/1817006545 shutdown_connections 2024-04-25T11:20:10.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a3e660640 1 --2- 172.21.15.106:0/1817006545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d1740 0x7f6a380d1b50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a3e660640 1 --2- 172.21.15.106:0/1817006545 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a380d0cf0 0x7f6a3806ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a3e660640 1 -- 172.21.15.106:0/1817006545 >> 172.21.15.106:0/1817006545 conn(0x7f6a3806c9f0 msgr2=0x7f6a3806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:10.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a3e660640 1 -- 172.21.15.106:0/1817006545 shutdown_connections 2024-04-25T11:20:10.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.952+0000 7f6a3e660640 1 -- 172.21.15.106:0/1817006545 wait complete. 2024-04-25T11:20:10.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 Processor -- start 2024-04-25T11:20:10.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 -- start start 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d0cf0 0x7f6a38161600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a380d1740 0x7f6a38142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3815d230 0x7f6a380b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3815cd40 con 0x7f6a380d0cf0 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3815cbc0 con 0x7f6a380d1740 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6a38073230 con 0x7f6a3815d230 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d0cf0 0x7f6a38161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d0cf0 0x7f6a38161600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41992/0 (socket says 172.21.15.106:41992) 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 -- 172.21.15.106:0/760205982 learned_addr learned my addr 172.21.15.106:0/760205982 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a377fe640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a380d1740 0x7f6a38142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 -- 172.21.15.106:0/760205982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3815d230 msgr2=0x7f6a380b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3815d230 0x7f6a380b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 -- 172.21.15.106:0/760205982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a380d1740 msgr2=0x7f6a38142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a380d1740 0x7f6a38142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 -- 172.21.15.106:0/760205982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a381549b0 con 0x7f6a380d0cf0 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a37fff640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d0cf0 0x7f6a38161600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800c820 tx=0x7f6a2800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a34ff9640 1 -- 172.21.15.106:0/760205982 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28015070 con 0x7f6a380d0cf0 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a34ff9640 1 -- 172.21.15.106:0/760205982 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a280079d0 con 0x7f6a380d0cf0 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a38154c90 con 0x7f6a380d0cf0 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a34ff9640 1 -- 172.21.15.106:0/760205982 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28007dc0 con 0x7f6a380d0cf0 2024-04-25T11:20:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.956+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a38155250 con 0x7f6a380d0cf0 2024-04-25T11:20:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.960+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a38050750 con 0x7f6a380d0cf0 2024-04-25T11:20:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.960+0000 7f6a34ff9640 1 -- 172.21.15.106:0/760205982 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6a28018070 con 0x7f6a380d0cf0 2024-04-25T11:20:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.960+0000 7f6a34ff9640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a2406f430 0x7f6a24071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.960+0000 7f6a377fe640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a2406f430 0x7f6a24071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.960+0000 7f6a34ff9640 1 -- 172.21.15.106:0/760205982 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6a2808f230 con 0x7f6a380d0cf0 2024-04-25T11:20:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.960+0000 7f6a377fe640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a2406f430 0x7f6a24071940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0b1000 tx=0x7f6a2c0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:10.964+0000 7f6a34ff9640 1 -- 172.21.15.106:0/760205982 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a2805c0a0 con 0x7f6a380d0cf0 2024-04-25T11:20:11.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.112+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6a380cd5e0 con 0x7f6a380d0cf0 2024-04-25T11:20:11.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.112+0000 7f6a34ff9640 1 -- 172.21.15.106:0/760205982 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f6a2805b9a0 con 0x7f6a380d0cf0 2024-04-25T11:20:11.112 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:11.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a2406f430 msgr2=0x7f6a24071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a2406f430 0x7f6a24071940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0b1000 tx=0x7f6a2c0a9070 comp rx=0 tx=0).stop 2024-04-25T11:20:11.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d0cf0 msgr2=0x7f6a38161600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d0cf0 0x7f6a38161600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800c820 tx=0x7f6a2800cce0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 shutdown_connections 2024-04-25T11:20:11.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3815d230 0x7f6a380b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a380d1740 0x7f6a38142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6a2406f430 0x7f6a24071940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 --2- 172.21.15.106:0/760205982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a380d0cf0 0x7f6a38161600 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 >> 172.21.15.106:0/760205982 conn(0x7f6a3806c9f0 msgr2=0x7f6a3815e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:11.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 shutdown_connections 2024-04-25T11:20:11.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.116+0000 7f6a3e660640 1 -- 172.21.15.106:0/760205982 wait complete. 2024-04-25T11:20:11.133 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:11.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d1b9bc640 1 Processor -- start 2024-04-25T11:20:11.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d1b9bc640 1 -- start start 2024-04-25T11:20:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d1b9bc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 0x7f8d14070460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d1b9bc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d140b5c10 con 0x7f8d140d1e50 2024-04-25T11:20:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d1b9bc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d14167da0 con 0x7f8d141642a0 2024-04-25T11:20:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d1b9bc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d141689b0 con 0x7f8d140d0bf0 2024-04-25T11:20:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d18f57640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8d140d1e50 0x7f8d1406fd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37282/0 (socket says 172.21.15.106:37282) 2024-04-25T11:20:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d18f57640 1 -- 172.21.15.106:0/1059398 learned_addr learned my addr 172.21.15.106:0/1059398 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471283287 0 0) 0x7f8d140b5c10 con 0x7f8d140d1e50 2024-04-25T11:20:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cfc003530 con 0x7f8d140d1e50 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d19f59640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 0x7f8d14070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3698594574 0 0) 0x7f8cfc003530 con 0x7f8d140d1e50 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 msgr2=0x7f8d14070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 0x7f8d14070460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 >> v1:172.21.15.134:6789/0 conn(0x7f8d141642a0 legacy=0x7f8d140d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d14168080 con 0x7f8d140d1e50 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f8d100016a0 con 0x7f8d140d1e50 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 >> v1:172.21.15.106:6789/0 conn(0x7f8d140d1e50 legacy=0x7f8d1406fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cfc0039f0 0x7f8cfc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 0x7f8cfc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cfc004d50 0x7f8cfc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141689b0 con 0x7f8cfc004d50 2024-04-25T11:20:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc00bc00 con 0x7f8cfc0039f0 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc00be00 con 0x7f8d140d0bf0 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d19f59640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cfc004d50 0x7f8cfc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d19758640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cfc0039f0 0x7f8cfc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.284+0000 7f8d18f57640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 0x7f8cfc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d19f59640 1 -- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 msgr2=0x7f8cfc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d19f59640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 0x7f8cfc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d19f59640 1 -- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cfc0039f0 msgr2=0x7f8cfc003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d19f59640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cfc0039f0 0x7f8cfc003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d19f59640 1 -- 172.21.15.106:0/1059398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc003530 con 0x7f8cfc004d50 2024-04-25T11:20:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d18f57640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140d0bf0 0x7f8cfc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:11.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d19f59640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cfc004d50 0x7f8cfc005220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ba00 tx=0x7f8d0401c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b053e7f0e7f420b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040ae070 con 0x7f8cfc004d50 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d040a9660 con 0x7f8cfc004d50 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d0bfff640 1 -- 172.21.15.106:0/1059398 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400b600 con 0x7f8cfc004d50 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/1059398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cfc004d50 msgr2=0x7f8cfc005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d1b9bc640 1 --2- 172.21.15.106:0/1059398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cfc004d50 0x7f8cfc005220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ba00 tx=0x7f8d0401c680 comp rx=0 tx=0).stop 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d19f59640 1 -- 172.21.15.106:0/1059398 reap_dead start 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/1059398 shutdown_connections 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/1059398 >> 172.21.15.106:0/1059398 conn(0x7f8d1406c9f0 msgr2=0x7f8d1406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/1059398 shutdown_connections 2024-04-25T11:20:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.288+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/1059398 wait complete. 2024-04-25T11:20:11.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 Processor -- start 2024-04-25T11:20:11.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 -- start start 2024-04-25T11:20:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14166890 0x7f8d140b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140b1500 0x7f8d14160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d14160c90 0x7f8d14157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d140d1000 con 0x7f8d14160c90 2024-04-25T11:20:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14168640 con 0x7f8d14166890 2024-04-25T11:20:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d140ce400 con 0x7f8d140b1500 2024-04-25T11:20:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d14160c90 0x7f8d14157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d14160c90 0x7f8d14157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42014/0 (socket says 172.21.15.106:42014) 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 -- 172.21.15.106:0/501417768 learned_addr learned my addr 172.21.15.106:0/501417768 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d18f57640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140b1500 0x7f8d14160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 -- 172.21.15.106:0/501417768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140b1500 msgr2=0x7f8d14160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19758640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14166890 0x7f8d140b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140b1500 0x7f8d14160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 -- 172.21.15.106:0/501417768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14166890 msgr2=0x7f8d140b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14166890 0x7f8d140b1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 -- 172.21.15.106:0/501417768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d140b2450 con 0x7f8d14160c90 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d19f59640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d14160c90 0x7f8d14157240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d0401cb60 tx=0x7f8cfc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d09ffb640 1 -- 172.21.15.106:0/501417768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040ae040 con 0x7f8d14160c90 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d09ffb640 1 -- 172.21.15.106:0/501417768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d040a9e70 con 0x7f8d14160c90 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d140b2730 con 0x7f8d14160c90 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d09ffb640 1 -- 172.21.15.106:0/501417768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400b600 con 0x7f8d14160c90 2024-04-25T11:20:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.292+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d140b0fe0 con 0x7f8d14160c90 2024-04-25T11:20:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.296+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d14050750 con 0x7f8d14160c90 2024-04-25T11:20:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.296+0000 7f8d09ffb640 1 -- 172.21.15.106:0/501417768 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8d040b1070 con 0x7f8d14160c90 2024-04-25T11:20:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.296+0000 7f8d09ffb640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8d0006f400 0x7f8d00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.296+0000 7f8d19758640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8d0006f400 0x7f8d00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.296+0000 7f8d09ffb640 1 -- 172.21.15.106:0/501417768 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8d04129ae0 con 0x7f8d14160c90 2024-04-25T11:20:11.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.296+0000 7f8d19758640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8d0006f400 0x7f8d00071910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d141661f0 tx=0x7f8d0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.300+0000 7f8d09ffb640 1 -- 172.21.15.106:0/501417768 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d040c1060 con 0x7f8d14160c90 2024-04-25T11:20:11.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.460+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d1406f620 con 0x7f8d14160c90 2024-04-25T11:20:11.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d09ffb640 1 -- 172.21.15.106:0/501417768 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f8d04016390 con 0x7f8d14160c90 2024-04-25T11:20:11.462 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:11.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8d0006f400 msgr2=0x7f8d00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8d0006f400 0x7f8d00071910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d141661f0 tx=0x7f8d0c009040 comp rx=0 tx=0).stop 2024-04-25T11:20:11.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d14160c90 msgr2=0x7f8d14157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d14160c90 0x7f8d14157240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d0401cb60 tx=0x7f8cfc005840 comp rx=0 tx=0).stop 2024-04-25T11:20:11.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 shutdown_connections 2024-04-25T11:20:11.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8d0006f400 0x7f8d00071910 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d14160c90 0x7f8d14157240 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140b1500 0x7f8d14160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 --2- 172.21.15.106:0/501417768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14166890 0x7f8d140b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 >> 172.21.15.106:0/501417768 conn(0x7f8d1406c9f0 msgr2=0x7f8d14072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:11.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 shutdown_connections 2024-04-25T11:20:11.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.464+0000 7f8d1b9bc640 1 -- 172.21.15.106:0/501417768 wait complete. 2024-04-25T11:20:11.484 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.628+0000 7fa18c7b0640 1 Processor -- start 2024-04-25T11:20:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.628+0000 7fa18c7b0640 1 -- start start 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18c7b0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa18415cae0 0x7fa184160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18c7b0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1840b6120 con 0x7fa1840cf6d0 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18c7b0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa184070920 con 0x7fa1840b52e0 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18c7b0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa1840ce2a0 con 0x7fa18415cae0 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa189d4b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa1840cf6d0 0x7fa1840b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37296/0 (socket says 172.21.15.106:37296) 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa189d4b640 1 -- 172.21.15.106:0/3678833868 learned_addr learned my addr 172.21.15.106:0/3678833868 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18a54c640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa18415cae0 0x7fa184160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2792806465 0 0) 0x7fa1840b6120 con 0x7fa1840cf6d0 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa170003530 con 0x7fa1840cf6d0 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 393086776 0 0) 0x7fa170003530 con 0x7fa1840cf6d0 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa18415cae0 msgr2=0x7fa184160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa18415cae0 0x7fa184160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 >> v1:172.21.15.134:6789/0 conn(0x7fa1840b52e0 legacy=0x7fa18406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa184070c00 con 0x7fa1840cf6d0 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fa174001680 con 0x7fa1840cf6d0 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 >> v1:172.21.15.106:6789/0 conn(0x7fa1840cf6d0 legacy=0x7fa1840b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa18415cae0 0x7fa1700039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa170003f70 0x7fa170004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa170005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840ce2a0 con 0x7fa170004d50 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa17000bc10 con 0x7fa170003f70 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa17000be10 con 0x7fa18415cae0 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18ad4d640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa170005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa189d4b640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa170003f70 0x7fa170004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18a54c640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa18415cae0 0x7fa1700039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18ad4d640 1 -- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa18415cae0 msgr2=0x7fa1700039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18ad4d640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa18415cae0 0x7fa1700039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18ad4d640 1 -- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa170003f70 msgr2=0x7fa170004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18ad4d640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa170003f70 0x7fa170004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18ad4d640 1 -- 172.21.15.106:0/3678833868 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa170003530 con 0x7fa170004d50 2024-04-25T11:20:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.632+0000 7fa18ad4d640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa170005160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa17c001e70 tx=0x7fa17c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d50fc58ca8b90fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c001a90 con 0x7fa170004d50 2024-04-25T11:20:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17c003b80 con 0x7fa170004d50 2024-04-25T11:20:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18954a640 1 -- 172.21.15.106:0/3678833868 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c005080 con 0x7fa170004d50 2024-04-25T11:20:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18c7b0640 1 -- 172.21.15.106:0/3678833868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 msgr2=0x7fa170005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18c7b0640 1 --2- 172.21.15.106:0/3678833868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa170005160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa17c001e70 tx=0x7fa17c01c780 comp rx=0 tx=0).stop 2024-04-25T11:20:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18ad4d640 1 -- 172.21.15.106:0/3678833868 reap_dead start 2024-04-25T11:20:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18c7b0640 1 -- 172.21.15.106:0/3678833868 shutdown_connections 2024-04-25T11:20:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18c7b0640 1 -- 172.21.15.106:0/3678833868 >> 172.21.15.106:0/3678833868 conn(0x7fa1840b3560 msgr2=0x7fa1840b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18c7b0640 1 -- 172.21.15.106:0/3678833868 shutdown_connections 2024-04-25T11:20:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18c7b0640 1 -- 172.21.15.106:0/3678833868 wait complete. 2024-04-25T11:20:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18c7b0640 1 Processor -- start 2024-04-25T11:20:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.636+0000 7fa18c7b0640 1 -- start start 2024-04-25T11:20:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa1841e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184157540 0x7fa184157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1840b4ad0 0x7fa1840b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840b4460 con 0x7fa170004d50 2024-04-25T11:20:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840b4740 con 0x7fa1840b4ad0 2024-04-25T11:20:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa184160fc0 con 0x7fa184157540 2024-04-25T11:20:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa1841e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa1841e6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42044/0 (socket says 172.21.15.106:42044) 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 -- 172.21.15.106:0/2197550392 learned_addr learned my addr 172.21.15.106:0/2197550392 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa189d4b640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184157540 0x7fa184157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18ad4d640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1840b4ad0 0x7fa1840b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 -- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184157540 msgr2=0x7fa184157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184157540 0x7fa184157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 -- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1840b4ad0 msgr2=0x7fa1840b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1840b4ad0 0x7fa1840b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 -- 172.21.15.106:0/2197550392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1840b5420 con 0x7fa170004d50 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa189d4b640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184157540 0x7fa184157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18a54c640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa1841e6fc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa1800115e0 tx=0x7fa180011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa17affd640 1 -- 172.21.15.106:0/2197550392 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18000f8f0 con 0x7fa170004d50 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa17affd640 1 -- 172.21.15.106:0/2197550392 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18000fa90 con 0x7fa170004d50 2024-04-25T11:20:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa17affd640 1 -- 172.21.15.106:0/2197550392 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18000fe40 con 0x7fa170004d50 2024-04-25T11:20:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1841f41b0 con 0x7fa170004d50 2024-04-25T11:20:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1841f4750 con 0x7fa170004d50 2024-04-25T11:20:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa184050750 con 0x7fa170004d50 2024-04-25T11:20:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa17affd640 1 -- 172.21.15.106:0/2197550392 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa180015460 con 0x7fa170004d50 2024-04-25T11:20:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa17affd640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa16806f400 0x7fa168071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa189d4b640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa16806f400 0x7fa168071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa17affd640 1 -- 172.21.15.106:0/2197550392 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa18008fc70 con 0x7fa170004d50 2024-04-25T11:20:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.640+0000 7fa189d4b640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa16806f400 0x7fa168071910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa174001060 tx=0x7fa1740019b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.644+0000 7fa17affd640 1 -- 172.21.15.106:0/2197550392 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1840b5420 con 0x7fa170004d50 2024-04-25T11:20:11.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa184069ad0 con 0x7fa170004d50 2024-04-25T11:20:11.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa17affd640 1 -- 172.21.15.106:0/2197550392 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fa180019030 con 0x7fa170004d50 2024-04-25T11:20:11.811 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa16806f400 msgr2=0x7fa168071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa16806f400 0x7fa168071910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa174001060 tx=0x7fa1740019b0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 msgr2=0x7fa1841e6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa1841e6fc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa1800115e0 tx=0x7fa180011aa0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 shutdown_connections 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1840b4ad0 0x7fa1840b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184157540 0x7fa184157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa16806f400 0x7fa168071910 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 --2- 172.21.15.106:0/2197550392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa170004d50 0x7fa1841e6fc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 >> 172.21.15.106:0/2197550392 conn(0x7fa1840b3560 msgr2=0x7fa18406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:11.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 shutdown_connections 2024-04-25T11:20:11.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:11.812+0000 7fa18c7b0640 1 -- 172.21.15.106:0/2197550392 wait complete. 2024-04-25T11:20:11.832 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:11.832 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:11.837 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:12.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8473519640 1 Processor -- start 2024-04-25T11:20:12.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8473519640 1 -- start start 2024-04-25T11:20:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8473519640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c0d1bd0 0x7f846c0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8473519640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846c0af370 con 0x7f846c06cd00 2024-04-25T11:20:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8473519640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846c0af650 con 0x7f846c15dd50 2024-04-25T11:20:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8473519640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f846c15e160 con 0x7f846c0d1bd0 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8470ab4640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f846c06cd00 0x7f846c0cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37308/0 (socket says 172.21.15.106:37308) 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8470ab4640 1 -- 172.21.15.106:0/3383484819 learned_addr learned my addr 172.21.15.106:0/3383484819 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f8471ab6640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c0d1bd0 0x7f846c0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4159009812 0 0) 0x7f846c0af370 con 0x7f846c06cd00 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8458003530 con 0x7f846c06cd00 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1326172785 0 0) 0x7f8458003530 con 0x7f846c06cd00 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c0d1bd0 msgr2=0x7f846c0d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f845ffff640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c0d1bd0 0x7f846c0d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 >> v1:172.21.15.134:6789/0 conn(0x7f846c15dd50 legacy=0x7f846c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c0af930 con 0x7f846c06cd00 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.024+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f84600016a0 con 0x7f846c06cd00 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 >> v1:172.21.15.106:6789/0 conn(0x7f846c06cd00 legacy=0x7f846c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84580039f0 0x7f8458003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f8458004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c0d1bd0 0x7f845800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f846c15e160 con 0x7f8458004810 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84580055d0 con 0x7f84580039f0 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f845800bf10 con 0x7f846c0d1bd0 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8470ab4640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f8458004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f84712b5640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84580039f0 0x7f8458003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8470ab4640 1 -- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c0d1bd0 msgr2=0x7f845800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8470ab4640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c0d1bd0 0x7f845800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8470ab4640 1 -- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84580039f0 msgr2=0x7f8458003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8470ab4640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84580039f0 0x7f8458003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8470ab4640 1 -- 172.21.15.106:0/3383484819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8458003530 con 0x7f8458004810 2024-04-25T11:20:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8470ab4640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f8458004c20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f846c073a10 tx=0x7f846001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d45d936947952e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460007d40 con 0x7f8458004810 2024-04-25T11:20:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84600a9070 con 0x7f8458004810 2024-04-25T11:20:12.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f845ffff640 1 -- 172.21.15.106:0/3383484819 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460005d40 con 0x7f8458004810 2024-04-25T11:20:12.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8473519640 1 -- 172.21.15.106:0/3383484819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 msgr2=0x7f8458004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8473519640 1 --2- 172.21.15.106:0/3383484819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f8458004c20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f846c073a10 tx=0x7f846001cbd0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8471ab6640 1 -- 172.21.15.106:0/3383484819 reap_dead start 2024-04-25T11:20:12.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8473519640 1 -- 172.21.15.106:0/3383484819 shutdown_connections 2024-04-25T11:20:12.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8473519640 1 -- 172.21.15.106:0/3383484819 >> 172.21.15.106:0/3383484819 conn(0x7f846c0b78b0 msgr2=0x7f846c0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:12.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8473519640 1 -- 172.21.15.106:0/3383484819 shutdown_connections 2024-04-25T11:20:12.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.028+0000 7f8473519640 1 -- 172.21.15.106:0/3383484819 wait complete. 2024-04-25T11:20:12.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 Processor -- start 2024-04-25T11:20:12.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 -- start start 2024-04-25T11:20:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f846c0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c14ffa0 0x7f846c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f846c0b5490 0x7f846c0b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f846c0d08b0 con 0x7f8458004810 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f846c0d1fe0 con 0x7f846c0b5490 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f846c0d0a30 con 0x7f846c14ffa0 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f846c0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f846c0b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42054/0 (socket says 172.21.15.106:42054) 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 -- 172.21.15.106:0/481088245 learned_addr learned my addr 172.21.15.106:0/481088245 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8470ab4640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c14ffa0 0x7f846c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 -- 172.21.15.106:0/481088245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c14ffa0 msgr2=0x7f846c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c14ffa0 0x7f846c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 -- 172.21.15.106:0/481088245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f846c0b5490 msgr2=0x7f846c0b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f846c0b5490 0x7f846c0b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 -- 172.21.15.106:0/481088245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c0b25a0 con 0x7f8458004810 2024-04-25T11:20:12.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f84712b5640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f846c0b3640 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f846400e900 tx=0x7f846400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f845dffb640 1 -- 172.21.15.106:0/481088245 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464003b90 con 0x7f8458004810 2024-04-25T11:20:12.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f845dffb640 1 -- 172.21.15.106:0/481088245 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8464018070 con 0x7f8458004810 2024-04-25T11:20:12.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c0b2880 con 0x7f8458004810 2024-04-25T11:20:12.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f845dffb640 1 -- 172.21.15.106:0/481088245 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846400c780 con 0x7f8458004810 2024-04-25T11:20:12.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.032+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c0b2e40 con 0x7f8458004810 2024-04-25T11:20:12.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.036+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c0b2300 con 0x7f8458004810 2024-04-25T11:20:12.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.036+0000 7f845dffb640 1 -- 172.21.15.106:0/481088245 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f846400c920 con 0x7f8458004810 2024-04-25T11:20:12.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.036+0000 7f845dffb640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f845006f430 0x7f8450071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.036+0000 7f845dffb640 1 -- 172.21.15.106:0/481088245 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f846408f4f0 con 0x7f8458004810 2024-04-25T11:20:12.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.036+0000 7f8470ab4640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f845006f430 0x7f8450071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.036+0000 7f8470ab4640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f845006f430 0x7f8450071940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f84600ab000 tx=0x7f84600a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.040+0000 7f845dffb640 1 -- 172.21.15.106:0/481088245 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f846c0b2300 con 0x7f8458004810 2024-04-25T11:20:12.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.192+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f846c070050 con 0x7f8458004810 2024-04-25T11:20:12.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.192+0000 7f845dffb640 1 -- 172.21.15.106:0/481088245 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16834 (secure 0 0 0) 0x7f846405c2c0 con 0x7f8458004810 2024-04-25T11:20:12.193 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:12.194 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:07.237284+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:12.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f845006f430 msgr2=0x7f8450071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f845006f430 0x7f8450071940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f84600ab000 tx=0x7f84600a7070 comp rx=0 tx=0).stop 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 msgr2=0x7f846c0b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f846c0b3640 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f846400e900 tx=0x7f846400edc0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 shutdown_connections 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f846c0b5490 0x7f846c0b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846c14ffa0 0x7f846c0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f845006f430 0x7f8450071940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 --2- 172.21.15.106:0/481088245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8458004810 0x7f846c0b3640 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 >> 172.21.15.106:0/481088245 conn(0x7f846c0b78b0 msgr2=0x7f846c070650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 shutdown_connections 2024-04-25T11:20:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.196+0000 7f8473519640 1 -- 172.21.15.106:0/481088245 wait complete. 2024-04-25T11:20:12.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:12.214 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:12.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dd58a5640 1 Processor -- start 2024-04-25T11:20:12.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dd58a5640 1 -- start start 2024-04-25T11:20:12.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dd58a5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7dd00d1080 0x7f7dd00d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dd58a5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd00d26c0 0x7f7dd00d4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dd58a5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd00d2ad0 con 0x7f7dd00d0c70 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dd58a5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd0164310 con 0x7f7dd00d1080 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dd58a5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd01645f0 con 0x7f7dd00d26c0 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dceffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7dd00d0c70 0x7f7dd0163890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37318/0 (socket says 172.21.15.106:37318) 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dceffd640 1 -- 172.21.15.106:0/4039975025 learned_addr learned my addr 172.21.15.106:0/4039975025 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcf7fe640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd00d26c0 0x7f7dd00d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dce7fc640 1 --2- 172.21.15.106:0/4039975025 >> v2:172.21.15.134:3300/0 conn(0x7f7dd00d1080 0x7f7dd00d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2282052190 0 0) 0x7f7dc0001120 con 0x7f7dd00d0c70 2024-04-25T11:20:12.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7db8003530 con 0x7f7dd00d0c70 2024-04-25T11:20:12.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1491783453 0 0) 0x7f7db8003530 con 0x7f7dd00d0c70 2024-04-25T11:20:12.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd00d26c0 msgr2=0x7f7dd00d4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd00d26c0 0x7f7dd00d4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 >> v2:172.21.15.134:3300/0 conn(0x7f7dd00d1080 msgr2=0x7f7dd00d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 --2- 172.21.15.106:0/4039975025 >> v2:172.21.15.134:3300/0 conn(0x7f7dd00d1080 0x7f7dd00d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd00d0470 con 0x7f7dd00d0c70 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f7dc0001850 con 0x7f7dd00d0c70 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 >> v1:172.21.15.106:6789/0 conn(0x7f7dd00d0c70 legacy=0x7f7dd0163890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7db8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd00d26c0 0x7f7db8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7db8004d50 0x7f7db8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0164310 con 0x7f7db80039c0 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01645f0 con 0x7f7db8004d50 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7db800bb00 con 0x7f7dd00d26c0 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.372+0000 7f7dceffd640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7db8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dce7fc640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd00d26c0 0x7f7db8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dcf7fe640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7db8004d50 0x7f7db8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dceffd640 1 -- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd00d26c0 msgr2=0x7f7db8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dceffd640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd00d26c0 0x7f7db8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dceffd640 1 -- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7db8004d50 msgr2=0x7f7db8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dceffd640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7db8004d50 0x7f7db8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dceffd640 1 -- 172.21.15.106:0/4039975025 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db8003530 con 0x7f7db80039c0 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dceffd640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7db8003e30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000c490 tx=0x7f7dc001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d010de82c6cc095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000c0e0 con 0x7f7db80039c0 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc00a9020 con 0x7f7db80039c0 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.376+0000 7f7dcdffb640 1 -- 172.21.15.106:0/4039975025 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a5b50 con 0x7f7db80039c0 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- 172.21.15.106:0/4039975025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 msgr2=0x7f7db8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 --2- 172.21.15.106:0/4039975025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7db8003e30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000c490 tx=0x7f7dc001cc70 comp rx=0 tx=0).stop 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dcf7fe640 1 -- 172.21.15.106:0/4039975025 reap_dead start 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- 172.21.15.106:0/4039975025 shutdown_connections 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- 172.21.15.106:0/4039975025 >> 172.21.15.106:0/4039975025 conn(0x7f7dd006daf0 msgr2=0x7f7dd006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- 172.21.15.106:0/4039975025 shutdown_connections 2024-04-25T11:20:12.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- 172.21.15.106:0/4039975025 wait complete. 2024-04-25T11:20:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 Processor -- start 2024-04-25T11:20:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- start start 2024-04-25T11:20:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7dd0155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd015c970 0x7f7dd01f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd01f9a10 0x7f7dd00c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00d4820 con 0x7f7db80039c0 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0160530 con 0x7f7dd015c970 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dd58a5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd00d49a0 con 0x7f7dd01f9a10 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dceffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7dd0155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dceffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7dd0155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42086/0 (socket says 172.21.15.106:42086) 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dceffd640 1 -- 172.21.15.106:0/864557874 learned_addr learned my addr 172.21.15.106:0/864557874 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dcf7fe640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd01f9a10 0x7f7dd00c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dce7fc640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd015c970 0x7f7dd01f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dceffd640 1 -- 172.21.15.106:0/864557874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd01f9a10 msgr2=0x7f7dd00c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dceffd640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd01f9a10 0x7f7dd00c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dceffd640 1 -- 172.21.15.106:0/864557874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd015c970 msgr2=0x7f7dd01f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dceffd640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd015c970 0x7f7dd01f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.380+0000 7f7dceffd640 1 -- 172.21.15.106:0/864557874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd015d0e0 con 0x7f7db80039c0 2024-04-25T11:20:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7dceffd640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7dd0155140 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7dc001caa0 tx=0x7f7db8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7db7fff640 1 -- 172.21.15.106:0/864557874 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0001130 con 0x7f7db80039c0 2024-04-25T11:20:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7db7fff640 1 -- 172.21.15.106:0/864557874 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc00a5950 con 0x7f7db80039c0 2024-04-25T11:20:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd015d3c0 con 0x7f7db80039c0 2024-04-25T11:20:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7db7fff640 1 -- 172.21.15.106:0/864557874 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a5d00 con 0x7f7db80039c0 2024-04-25T11:20:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd015d980 con 0x7f7db80039c0 2024-04-25T11:20:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd0050750 con 0x7f7db80039c0 2024-04-25T11:20:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7db7fff640 1 -- 172.21.15.106:0/864557874 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7dc00047f0 con 0x7f7db80039c0 2024-04-25T11:20:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7db7fff640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7dbc06f400 0x7f7dbc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7db7fff640 1 -- 172.21.15.106:0/864557874 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7dc0128350 con 0x7f7db80039c0 2024-04-25T11:20:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7dce7fc640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7dbc06f400 0x7f7dbc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.384+0000 7f7dce7fc640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7dbc06f400 0x7f7dbc071910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009160 tx=0x7f7dc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.388+0000 7f7db7fff640 1 -- 172.21.15.106:0/864557874 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7dc00b4070 con 0x7f7db80039c0 2024-04-25T11:20:12.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7dd00d3bc0 con 0x7f7db80039c0 2024-04-25T11:20:12.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7db7fff640 1 -- 172.21.15.106:0/864557874 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f7dd00d2ad0 con 0x7f7db80039c0 2024-04-25T11:20:12.547 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7dbc06f400 msgr2=0x7f7dbc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7dbc06f400 0x7f7dbc071910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009160 tx=0x7f7dc4008040 comp rx=0 tx=0).stop 2024-04-25T11:20:12.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 msgr2=0x7f7dd0155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7dd0155140 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7dc001caa0 tx=0x7f7db8005840 comp rx=0 tx=0).stop 2024-04-25T11:20:12.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 shutdown_connections 2024-04-25T11:20:12.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dd01f9a10 0x7f7dd00c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd015c970 0x7f7dd01f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7dbc06f400 0x7f7dbc071910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 --2- 172.21.15.106:0/864557874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80039c0 0x7f7dd0155140 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 >> 172.21.15.106:0/864557874 conn(0x7f7dd006daf0 msgr2=0x7f7dd0165e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:12.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 shutdown_connections 2024-04-25T11:20:12.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.548+0000 7f7dd58a5640 1 -- 172.21.15.106:0/864557874 wait complete. 2024-04-25T11:20:12.568 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:12.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a618d3640 1 Processor -- start 2024-04-25T11:20:12.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a618d3640 1 -- start start 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a618d3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2a5c0d0cf0 0x7f2a5c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a618d3640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5c0b68e0 con 0x7f2a5c0d1740 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a618d3640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5c161210 con 0x7f2a5c15d230 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a618d3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c0b6290 con 0x7f2a5c0d0cf0 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a5a7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2a5c0d1740 0x7f2a5c0d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37328/0 (socket says 172.21.15.106:37328) 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a5a7fc640 1 -- 172.21.15.106:0/602518477 learned_addr learned my addr 172.21.15.106:0/602518477 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a5affd640 1 --2- 172.21.15.106:0/602518477 >> v2:172.21.15.134:3300/0 conn(0x7f2a5c0d0cf0 0x7f2a5c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2617693403 0 0) 0x7f2a5c0b68e0 con 0x7f2a5c0d1740 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a44003530 con 0x7f2a5c0d1740 2024-04-25T11:20:12.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1969591837 0 0) 0x7f2a44003530 con 0x7f2a5c0d1740 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 >> v2:172.21.15.134:3300/0 conn(0x7f2a5c0d0cf0 msgr2=0x7f2a5c06ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 --2- 172.21.15.106:0/602518477 >> v2:172.21.15.134:3300/0 conn(0x7f2a5c0d0cf0 0x7f2a5c06ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 >> v1:172.21.15.134:6789/0 conn(0x7f2a5c15d230 legacy=0x7f2a5c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c0b6570 con 0x7f2a5c0d1740 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f2a4c001680 con 0x7f2a5c0d1740 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.720+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 >> v1:172.21.15.106:6789/0 conn(0x7f2a5c0d1740 legacy=0x7f2a5c0d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a440039c0 0x7f2a44003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a44004810 0x7f2a44004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a44005360 0x7f2a4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0b6290 con 0x7f2a44005360 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4400c290 con 0x7f2a440039c0 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4400c490 con 0x7f2a44004810 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5b7fe640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a44005360 0x7f2a4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5affd640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a440039c0 0x7f2a44003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5a7fc640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a44004810 0x7f2a44004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5b7fe640 1 -- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a44004810 msgr2=0x7f2a44004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5b7fe640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a44004810 0x7f2a44004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5b7fe640 1 -- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a440039c0 msgr2=0x7f2a44003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5b7fe640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a440039c0 0x7f2a44003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5b7fe640 1 -- 172.21.15.106:0/602518477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a44003530 con 0x7f2a44005360 2024-04-25T11:20:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5a7fc640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a44004810 0x7f2a44004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:12.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5b7fe640 1 -- 172.21.15.106:0/602518477 reap_dead start 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a5b7fe640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a44005360 0x7f2a4400b9d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2a54001e70 tx=0x7f2a5401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da36b13d109e87d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54001a90 con 0x7f2a44005360 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a54003ba0 con 0x7f2a44005360 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a59ffb640 1 -- 172.21.15.106:0/602518477 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a540050d0 con 0x7f2a44005360 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a618d3640 1 -- 172.21.15.106:0/602518477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a44005360 msgr2=0x7f2a4400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a618d3640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a44005360 0x7f2a4400b9d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2a54001e70 tx=0x7f2a5401c780 comp rx=0 tx=0).stop 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a618d3640 1 -- 172.21.15.106:0/602518477 shutdown_connections 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a618d3640 1 --2- 172.21.15.106:0/602518477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a44005360 0x7f2a4400b9d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a618d3640 1 -- 172.21.15.106:0/602518477 >> 172.21.15.106:0/602518477 conn(0x7f2a5c06c9f0 msgr2=0x7f2a5c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a618d3640 1 -- 172.21.15.106:0/602518477 shutdown_connections 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.724+0000 7f2a618d3640 1 -- 172.21.15.106:0/602518477 wait complete. 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 Processor -- start 2024-04-25T11:20:12.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 -- start start 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a44005360 0x7f2a5c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a5c155bb0 0x7f2a5c153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a5c15eb80 0x7f2a5c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c06ea20 con 0x7f2a5c15eb80 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c15cdd0 con 0x7f2a44005360 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c06eba0 con 0x7f2a5c155bb0 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a5c15eb80 0x7f2a5c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a5c15eb80 0x7f2a5c1544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42108/0 (socket says 172.21.15.106:42108) 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 -- 172.21.15.106:0/2928246877 learned_addr learned my addr 172.21.15.106:0/2928246877 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5a7fc640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a5c155bb0 0x7f2a5c153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5affd640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a44005360 0x7f2a5c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 -- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a5c155bb0 msgr2=0x7f2a5c153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a5c155bb0 0x7f2a5c153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 -- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a44005360 msgr2=0x7f2a5c155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a44005360 0x7f2a5c155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 -- 172.21.15.106:0/2928246877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c1606c0 con 0x7f2a5c15eb80 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5affd640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a44005360 0x7f2a5c155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a5b7fe640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a5c15eb80 0x7f2a5c1544c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a5401d020 tx=0x7f2a54019520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a43fff640 1 -- 172.21.15.106:0/2928246877 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54001a90 con 0x7f2a5c15eb80 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a43fff640 1 -- 172.21.15.106:0/2928246877 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a54001260 con 0x7f2a5c15eb80 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c160960 con 0x7f2a5c15eb80 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a43fff640 1 -- 172.21.15.106:0/2928246877 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54006b60 con 0x7f2a5c15eb80 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.728+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c15be20 con 0x7f2a5c15eb80 2024-04-25T11:20:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.732+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c050750 con 0x7f2a5c15eb80 2024-04-25T11:20:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.732+0000 7f2a43fff640 1 -- 172.21.15.106:0/2928246877 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2a54006d00 con 0x7f2a5c15eb80 2024-04-25T11:20:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.732+0000 7f2a43fff640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2a4806f430 0x7f2a48071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.732+0000 7f2a5affd640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2a4806f430 0x7f2a48071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.732+0000 7f2a43fff640 1 -- 172.21.15.106:0/2928246877 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2a541287e0 con 0x7f2a5c15eb80 2024-04-25T11:20:12.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.732+0000 7f2a5affd640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2a4806f430 0x7f2a48071940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0721f0 tx=0x7f2a50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.736+0000 7f2a43fff640 1 -- 172.21.15.106:0/2928246877 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2a54016490 con 0x7f2a5c15eb80 2024-04-25T11:20:12.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.883+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2a5c15eff0 con 0x7f2a5c15eb80 2024-04-25T11:20:12.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.883+0000 7f2a43fff640 1 -- 172.21.15.106:0/2928246877 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f2a540f5650 con 0x7f2a5c15eb80 2024-04-25T11:20:12.883 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.883+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2a4806f430 msgr2=0x7f2a48071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.883+0000 7f2a618d3640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2a4806f430 0x7f2a48071940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0721f0 tx=0x7f2a50008040 comp rx=0 tx=0).stop 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.883+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a5c15eb80 msgr2=0x7f2a5c1544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.883+0000 7f2a618d3640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a5c15eb80 0x7f2a5c1544c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a5401d020 tx=0x7f2a54019520 comp rx=0 tx=0).stop 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.883+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 shutdown_connections 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.883+0000 7f2a618d3640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2a4806f430 0x7f2a48071940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.887+0000 7f2a618d3640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a5c15eb80 0x7f2a5c1544c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.887+0000 7f2a618d3640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a5c155bb0 0x7f2a5c153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.887+0000 7f2a618d3640 1 --2- 172.21.15.106:0/2928246877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a44005360 0x7f2a5c155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.887+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 >> 172.21.15.106:0/2928246877 conn(0x7f2a5c06c9f0 msgr2=0x7f2a5c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:12.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.887+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 shutdown_connections 2024-04-25T11:20:12.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:12.887+0000 7f2a618d3640 1 -- 172.21.15.106:0/2928246877 wait complete. 2024-04-25T11:20:12.903 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:13.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.039+0000 7fad42e80640 1 Processor -- start 2024-04-25T11:20:13.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.039+0000 7fad42e80640 1 -- start start 2024-04-25T11:20:13.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad42e80640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fad3c164c30 0x7fad3c16b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad42e80640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3c0d0da0 con 0x7fad3c0b87d0 2024-04-25T11:20:13.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad42e80640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3c09cbe0 con 0x7fad3c164110 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad42e80640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fad3c09cf50 con 0x7fad3c164c30 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad40c1c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fad3c0b87d0 0x7fad3c0d1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37344/0 (socket says 172.21.15.106:37344) 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad40c1c640 1 -- 172.21.15.106:0/719697286 learned_addr learned my addr 172.21.15.106:0/719697286 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad4141d640 1 --2- 172.21.15.106:0/719697286 >> v2:172.21.15.134:3300/0 conn(0x7fad3c164c30 0x7fad3c16b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1102405387 0 0) 0x7fad3c0d0da0 con 0x7fad3c0b87d0 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad24003530 con 0x7fad3c0b87d0 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2340649475 0 0) 0x7fad24003530 con 0x7fad3c0b87d0 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 >> v2:172.21.15.134:3300/0 conn(0x7fad3c164c30 msgr2=0x7fad3c16b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 --2- 172.21.15.106:0/719697286 >> v2:172.21.15.134:3300/0 conn(0x7fad3c164c30 0x7fad3c16b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 >> v1:172.21.15.134:6789/0 conn(0x7fad3c164110 legacy=0x7fad3c164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c0d2370 con 0x7fad3c0b87d0 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fad2c001800 con 0x7fad3c0b87d0 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 >> v1:172.21.15.106:6789/0 conn(0x7fad3c0b87d0 legacy=0x7fad3c0d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad240039c0 0x7fad24003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad24004810 0x7fad24004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad24005260 0x7fad2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c09cf50 con 0x7fad240039c0 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad2400c1f0 con 0x7fad24005260 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad2400c3f0 con 0x7fad24004810 2024-04-25T11:20:13.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad40c1c640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad240039c0 0x7fad24003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad4141d640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad24005260 0x7fad2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad40c1c640 1 -- 172.21.15.106:0/719697286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad24004810 msgr2=0x7fad24004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:13.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad40c1c640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad24004810 0x7fad24004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad40c1c640 1 -- 172.21.15.106:0/719697286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad24005260 msgr2=0x7fad2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:13.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad40c1c640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad24005260 0x7fad2400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad40c1c640 1 -- 172.21.15.106:0/719697286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad24003530 con 0x7fad240039c0 2024-04-25T11:20:13.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad4141d640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad24005260 0x7fad2400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:13.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.043+0000 7fad4141d640 1 -- 172.21.15.106:0/719697286 reap_dead start 2024-04-25T11:20:13.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad40c1c640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad240039c0 0x7fad24003e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c210 tx=0x7fad2c01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6482b5eff3441c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c007cf0 con 0x7fad240039c0 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c0a9020 con 0x7fad240039c0 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad3b7fe640 1 -- 172.21.15.106:0/719697286 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0072f0 con 0x7fad240039c0 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 -- 172.21.15.106:0/719697286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad240039c0 msgr2=0x7fad24003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad240039c0 0x7fad24003e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c210 tx=0x7fad2c01cc00 comp rx=0 tx=0).stop 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 -- 172.21.15.106:0/719697286 shutdown_connections 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 --2- 172.21.15.106:0/719697286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad240039c0 0x7fad24003e30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 -- 172.21.15.106:0/719697286 >> 172.21.15.106:0/719697286 conn(0x7fad3c0d00e0 msgr2=0x7fad3c0d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 -- 172.21.15.106:0/719697286 shutdown_connections 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 -- 172.21.15.106:0/719697286 wait complete. 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 Processor -- start 2024-04-25T11:20:13.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.047+0000 7fad42e80640 1 -- start start 2024-04-25T11:20:13.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad240039c0 0x7fad3c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad3c0b0fd0 0x7fad3c0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad3c0b1920 0x7fad3c1fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0d3410 con 0x7fad3c0b1920 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0d3290 con 0x7fad3c0b0fd0 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c06df10 con 0x7fad240039c0 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad3c0b1920 0x7fad3c1fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad3c0b1920 0x7fad3c1fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42128/0 (socket says 172.21.15.106:42128) 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 -- 172.21.15.106:0/2490458987 learned_addr learned my addr 172.21.15.106:0/2490458987 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad40c1c640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad240039c0 0x7fad3c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad3bfff640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad3c0b0fd0 0x7fad3c0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 -- 172.21.15.106:0/2490458987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad240039c0 msgr2=0x7fad3c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad240039c0 0x7fad3c156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 -- 172.21.15.106:0/2490458987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad3c0b0fd0 msgr2=0x7fad3c0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad3c0b0fd0 0x7fad3c0b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 -- 172.21.15.106:0/2490458987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c0b0510 con 0x7fad3c0b1920 2024-04-25T11:20:13.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad4141d640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad3c0b1920 0x7fad3c1fd110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad3c051150 tx=0x7fad3400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad397fa640 1 -- 172.21.15.106:0/2490458987 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34016620 con 0x7fad3c0b1920 2024-04-25T11:20:13.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad397fa640 1 -- 172.21.15.106:0/2490458987 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3401b070 con 0x7fad3c0b1920 2024-04-25T11:20:13.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad397fa640 1 -- 172.21.15.106:0/2490458987 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad340169c0 con 0x7fad3c0b1920 2024-04-25T11:20:13.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c162c90 con 0x7fad3c0b1920 2024-04-25T11:20:13.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c0b2050 con 0x7fad3c0b1920 2024-04-25T11:20:13.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3c162a00 con 0x7fad3c0b1920 2024-04-25T11:20:13.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad397fa640 1 -- 172.21.15.106:0/2490458987 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad34016b60 con 0x7fad3c0b1920 2024-04-25T11:20:13.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.051+0000 7fad397fa640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fad2806f430 0x7fad28071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.055+0000 7fad397fa640 1 -- 172.21.15.106:0/2490458987 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fad3408fe80 con 0x7fad3c0b1920 2024-04-25T11:20:13.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.055+0000 7fad40c1c640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fad2806f430 0x7fad28071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.055+0000 7fad40c1c640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fad2806f430 0x7fad28071940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0aa000 tx=0x7fad2c0b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.059+0000 7fad397fa640 1 -- 172.21.15.106:0/2490458987 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad3405ce10 con 0x7fad3c0b1920 2024-04-25T11:20:13.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.203+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad3c0b22b0 con 0x7fad3c0b1920 2024-04-25T11:20:13.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad397fa640 1 -- 172.21.15.106:0/2490458987 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fad3405c710 con 0x7fad3c0b1920 2024-04-25T11:20:13.206 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:13.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fad2806f430 msgr2=0x7fad28071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fad2806f430 0x7fad28071940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0aa000 tx=0x7fad2c0b0040 comp rx=0 tx=0).stop 2024-04-25T11:20:13.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad3c0b1920 msgr2=0x7fad3c1fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad3c0b1920 0x7fad3c1fd110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad3c051150 tx=0x7fad3400f520 comp rx=0 tx=0).stop 2024-04-25T11:20:13.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 shutdown_connections 2024-04-25T11:20:13.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fad2806f430 0x7fad28071940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad3c0b1920 0x7fad3c1fd110 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad3c0b0fd0 0x7fad3c0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 --2- 172.21.15.106:0/2490458987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad240039c0 0x7fad3c156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 >> 172.21.15.106:0/2490458987 conn(0x7fad3c0d00e0 msgr2=0x7fad3c0b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:13.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 shutdown_connections 2024-04-25T11:20:13.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.207+0000 7fad42e80640 1 -- 172.21.15.106:0/2490458987 wait complete. 2024-04-25T11:20:13.227 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:13.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b40c3640 1 Processor -- start 2024-04-25T11:20:13.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b40c3640 1 -- start start 2024-04-25T11:20:13.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b40c3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f34ac0cf0e0 0x7f34ac09bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b40c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac09c140 0x7f34ac0d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b40c3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f34ac0d1680 0x7f34ac0d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b40c3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f34ac09ce70 con 0x7f34ac0cf0e0 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b40c3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f34ac0b8ce0 con 0x7f34ac0d1680 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b40c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac0b46f0 con 0x7f34ac09c140 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b1e5f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f34ac0cf0e0 0x7f34ac09bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b1e5f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f34ac0cf0e0 0x7f34ac09bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42130/0 (socket says 172.21.15.106:42130) 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b1e5f640 1 -- 172.21.15.106:0/4224776150 learned_addr learned my addr 172.21.15.106:0/4224776150 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b165e640 1 --2- 172.21.15.106:0/4224776150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac09c140 0x7f34ac0d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b2660640 1 --2- 172.21.15.106:0/4224776150 >> v2:172.21.15.134:3300/0 conn(0x7f34ac0d1680 0x7f34ac0d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b1e5f640 1 -- 172.21.15.106:0/4224776150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac09c140 msgr2=0x7f34ac0d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b1e5f640 1 --2- 172.21.15.106:0/4224776150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac09c140 0x7f34ac0d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b1e5f640 1 -- 172.21.15.106:0/4224776150 >> v2:172.21.15.134:3300/0 conn(0x7f34ac0d1680 msgr2=0x7f34ac0d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b1e5f640 1 --2- 172.21.15.106:0/4224776150 >> v2:172.21.15.134:3300/0 conn(0x7f34ac0d1680 0x7f34ac0d1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.375+0000 7f34b1e5f640 1 -- 172.21.15.106:0/4224776150 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac0cf4f0 con 0x7f34ac0cf0e0 2024-04-25T11:20:13.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b1e5f640 1 --2- 172.21.15.106:0/4224776150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac0cf0e0 0x7f34ac09bc00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0cff20 tx=0x7f34a401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd95790bea2984d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b0e5d640 1 -- 172.21.15.106:0/4224776150 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a40a58b0 con 0x7f34ac0cf0e0 2024-04-25T11:20:13.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b0e5d640 1 -- 172.21.15.106:0/4224776150 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a40a5a50 con 0x7f34ac0cf0e0 2024-04-25T11:20:13.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b0e5d640 1 -- 172.21.15.106:0/4224776150 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a40a5e20 con 0x7f34ac0cf0e0 2024-04-25T11:20:13.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 -- 172.21.15.106:0/4224776150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac0cf0e0 msgr2=0x7f34ac09bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 --2- 172.21.15.106:0/4224776150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac0cf0e0 0x7f34ac09bc00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0cff20 tx=0x7f34a401ca30 comp rx=0 tx=0).stop 2024-04-25T11:20:13.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 -- 172.21.15.106:0/4224776150 shutdown_connections 2024-04-25T11:20:13.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 --2- 172.21.15.106:0/4224776150 >> v2:172.21.15.134:3300/0 conn(0x7f34ac0d1680 0x7f34ac0d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 --2- 172.21.15.106:0/4224776150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac09c140 0x7f34ac0d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 --2- 172.21.15.106:0/4224776150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac0cf0e0 0x7f34ac09bc00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 -- 172.21.15.106:0/4224776150 >> 172.21.15.106:0/4224776150 conn(0x7f34ac0b5060 msgr2=0x7f34ac0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:13.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 -- 172.21.15.106:0/4224776150 shutdown_connections 2024-04-25T11:20:13.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.379+0000 7f34b40c3640 1 -- 172.21.15.106:0/4224776150 wait complete. 2024-04-25T11:20:13.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 Processor -- start 2024-04-25T11:20:13.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 -- start start 2024-04-25T11:20:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac09c140 0x7f34ac156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ac0cf0e0 0x7f34ac167310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac0d1680 0x7f34ac167850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac09c550 con 0x7f34ac09c140 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac09c870 con 0x7f34ac0cf0e0 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac0b8e80 con 0x7f34ac0d1680 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac09c140 0x7f34ac156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac09c140 0x7f34ac156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42132/0 (socket says 172.21.15.106:42132) 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 -- 172.21.15.106:0/1623124115 learned_addr learned my addr 172.21.15.106:0/1623124115 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b165e640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ac0cf0e0 0x7f34ac167310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b2660640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac0d1680 0x7f34ac167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 -- 172.21.15.106:0/1623124115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac0d1680 msgr2=0x7f34ac167850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac0d1680 0x7f34ac167850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 -- 172.21.15.106:0/1623124115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ac0cf0e0 msgr2=0x7f34ac167310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ac0cf0e0 0x7f34ac167310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 -- 172.21.15.106:0/1623124115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac0b8050 con 0x7f34ac09c140 2024-04-25T11:20:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b1e5f640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac09c140 0x7f34ac156bb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f34a401b0e0 tx=0x7f34a40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34aa7fc640 1 -- 172.21.15.106:0/1623124115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a40a58b0 con 0x7f34ac09c140 2024-04-25T11:20:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34aa7fc640 1 -- 172.21.15.106:0/1623124115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a40b8470 con 0x7f34ac09c140 2024-04-25T11:20:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac1647e0 con 0x7f34ac09c140 2024-04-25T11:20:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34aa7fc640 1 -- 172.21.15.106:0/1623124115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4009510 con 0x7f34ac09c140 2024-04-25T11:20:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.383+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac0b6510 con 0x7f34ac09c140 2024-04-25T11:20:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.387+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac050750 con 0x7f34ac09c140 2024-04-25T11:20:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.387+0000 7f34aa7fc640 1 -- 172.21.15.106:0/1623124115 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f34a40c0070 con 0x7f34ac09c140 2024-04-25T11:20:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.387+0000 7f34aa7fc640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f349806f430 0x7f3498071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.387+0000 7f34b165e640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f349806f430 0x7f3498071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.387+0000 7f34aa7fc640 1 -- 172.21.15.106:0/1623124115 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f34a4129ff0 con 0x7f34ac09c140 2024-04-25T11:20:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.387+0000 7f34b165e640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f349806f430 0x7f3498071940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0d1e50 tx=0x7f34a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.391+0000 7f34aa7fc640 1 -- 172.21.15.106:0/1623124115 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34a4016620 con 0x7f34ac09c140 2024-04-25T11:20:13.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.563+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34ac0759a0 con 0x7f34ac09c140 2024-04-25T11:20:13.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.563+0000 7f34aa7fc640 1 -- 172.21.15.106:0/1623124115 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f34a40f6fb0 con 0x7f34ac09c140 2024-04-25T11:20:13.564 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f349806f430 msgr2=0x7f3498071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f349806f430 0x7f3498071940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0d1e50 tx=0x7f34a0008040 comp rx=0 tx=0).stop 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac09c140 msgr2=0x7f34ac156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac09c140 0x7f34ac156bb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f34a401b0e0 tx=0x7f34a40197d0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 shutdown_connections 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34ac0d1680 0x7f34ac167850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ac0cf0e0 0x7f34ac167310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f349806f430 0x7f3498071940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 --2- 172.21.15.106:0/1623124115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac09c140 0x7f34ac156bb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 >> 172.21.15.106:0/1623124115 conn(0x7f34ac0b5060 msgr2=0x7f34ac0b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 shutdown_connections 2024-04-25T11:20:13.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.567+0000 7f34b40c3640 1 -- 172.21.15.106:0/1623124115 wait complete. 2024-04-25T11:20:13.586 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-25T11:20:13.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:13.586 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T11:20:13.599 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:13.599 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:20:13.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:13.599 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T11:20:13.647 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:20:13.647 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:13.653 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:20:13.654 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:13.701 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.851+0000 7fa21087b640 1 Processor -- start 2024-04-25T11:20:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.851+0000 7fa21087b640 1 -- start start 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa21087b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c156f90 0x7fa20c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa21087b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20c1573a0 con 0x7fa20c0b5a80 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa21087b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20c15d220 con 0x7fa20c06fea0 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa21087b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c0d1f90 con 0x7fa20c156f90 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa20c0b5a80 0x7fa20c0b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37360/0 (socket says 172.21.15.106:37360) 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20ad9d640 1 -- 172.21.15.106:0/2857393773 learned_addr learned my addr 172.21.15.106:0/2857393773 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20a59c640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c156f90 0x7fa20c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860765959 0 0) 0x7fa20c1573a0 con 0x7fa20c0b5a80 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1f8003530 con 0x7fa20c0b5a80 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2329936630 0 0) 0x7fa1f8003530 con 0x7fa20c0b5a80 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c156f90 msgr2=0x7fa20c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c156f90 0x7fa20c0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 >> v1:172.21.15.134:6789/0 conn(0x7fa20c06fea0 legacy=0x7fa20c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c15d500 con 0x7fa20c0b5a80 2024-04-25T11:20:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fa204001a00 con 0x7fa20c0b5a80 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 >> v1:172.21.15.106:6789/0 conn(0x7fa20c0b5a80 legacy=0x7fa20c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1f80039f0 0x7fa1f8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1f8004810 0x7fa1f8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c156f90 0x7fa1f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c0d1f90 con 0x7fa1f8004810 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f800bb00 con 0x7fa1f80039f0 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f800bd00 con 0x7fa20c156f90 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa209d9b640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1f8004810 0x7fa1f8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20ad9d640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c156f90 0x7fa1f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa209d9b640 1 -- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c156f90 msgr2=0x7fa1f8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa209d9b640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c156f90 0x7fa1f8005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa209d9b640 1 -- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1f80039f0 msgr2=0x7fa1f8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa20a59c640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1f80039f0 0x7fa1f8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa209d9b640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1f80039f0 0x7fa1f8003e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.855+0000 7fa209d9b640 1 -- 172.21.15.106:0/2857393773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f8003530 con 0x7fa1f8004810 2024-04-25T11:20:13.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa209d9b640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1f8004810 0x7fa1f8004c20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4001750 tx=0x7fa1f401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babfe444ad4a17b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40013a0 con 0x7fa1f8004810 2024-04-25T11:20:13.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f40052f0 con 0x7fa1f8004810 2024-04-25T11:20:13.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa20959a640 1 -- 172.21.15.106:0/2857393773 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4003840 con 0x7fa1f8004810 2024-04-25T11:20:13.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa21087b640 1 -- 172.21.15.106:0/2857393773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1f8004810 msgr2=0x7fa1f8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa21087b640 1 --2- 172.21.15.106:0/2857393773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1f8004810 0x7fa1f8004c20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4001750 tx=0x7fa1f401c3e0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa20ad9d640 1 -- 172.21.15.106:0/2857393773 reap_dead start 2024-04-25T11:20:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa21087b640 1 -- 172.21.15.106:0/2857393773 shutdown_connections 2024-04-25T11:20:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa21087b640 1 -- 172.21.15.106:0/2857393773 >> 172.21.15.106:0/2857393773 conn(0x7fa20c06c7c0 msgr2=0x7fa20c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa21087b640 1 -- 172.21.15.106:0/2857393773 shutdown_connections 2024-04-25T11:20:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa21087b640 1 -- 172.21.15.106:0/2857393773 wait complete. 2024-04-25T11:20:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa21087b640 1 Processor -- start 2024-04-25T11:20:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.859+0000 7fa21087b640 1 -- start start 2024-04-25T11:20:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa20c1f9900 0x7fa20c1486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c1454a0 0x7fa20c0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa20c0b25e0 0x7fa20c1f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c15dac0 con 0x7fa20c1f9900 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c0b4810 con 0x7fa20c0b25e0 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c15ced0 con 0x7fa20c1454a0 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa20c1f9900 0x7fa20c1486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa20c1f9900 0x7fa20c1486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42142/0 (socket says 172.21.15.106:42142) 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 -- 172.21.15.106:0/470150036 learned_addr learned my addr 172.21.15.106:0/470150036 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa209d9b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c1454a0 0x7fa20c0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20ad9d640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa20c0b25e0 0x7fa20c1f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 -- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c1454a0 msgr2=0x7fa20c0b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c1454a0 0x7fa20c0b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 -- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa20c0b25e0 msgr2=0x7fa20c1f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa20c0b25e0 0x7fa20c1f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 -- 172.21.15.106:0/470150036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c0b29f0 con 0x7fa20c1f9900 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa209d9b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c1454a0 0x7fa20c0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20ad9d640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa20c0b25e0 0x7fa20c1f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa20a59c640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa20c1f9900 0x7fa20c1486f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc010890 tx=0x7fa1fc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa202ffd640 1 -- 172.21.15.106:0/470150036 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00f040 con 0x7fa20c1f9900 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa202ffd640 1 -- 172.21.15.106:0/470150036 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00eb60 con 0x7fa20c1f9900 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c1f9ea0 con 0x7fa20c1f9900 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa202ffd640 1 -- 172.21.15.106:0/470150036 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc01c400 con 0x7fa20c1f9900 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c1fa460 con 0x7fa20c1f9900 2024-04-25T11:20:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20c0b39e0 con 0x7fa20c1f9900 2024-04-25T11:20:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.863+0000 7fa202ffd640 1 -- 172.21.15.106:0/470150036 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa1fc01c5a0 con 0x7fa20c1f9900 2024-04-25T11:20:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.867+0000 7fa202ffd640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa1ec06f400 0x7fa1ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.867+0000 7fa209d9b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa1ec06f400 0x7fa1ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.867+0000 7fa202ffd640 1 -- 172.21.15.106:0/470150036 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa1fc08f8c0 con 0x7fa20c1f9900 2024-04-25T11:20:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.867+0000 7fa209d9b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa1ec06f400 0x7fa1ec071910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8004790 tx=0x7fa1f40011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:13.871+0000 7fa202ffd640 1 -- 172.21.15.106:0/470150036 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa20c0b29f0 con 0x7fa20c1f9900 2024-04-25T11:20:14.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.019+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa20c0b6520 con 0x7fa20c1f9900 2024-04-25T11:20:14.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.019+0000 7fa202ffd640 1 -- 172.21.15.106:0/470150036 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16834 (secure 0 0 0) 0x7fa1fc018030 con 0x7fa20c1f9900 2024-04-25T11:20:14.023 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:14.023 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:07.237284+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:14.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa1ec06f400 msgr2=0x7fa1ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa1ec06f400 0x7fa1ec071910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8004790 tx=0x7fa1f40011a0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa20c1f9900 msgr2=0x7fa20c1486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa20c1f9900 0x7fa20c1486f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc010890 tx=0x7fa1fc010d50 comp rx=0 tx=0).stop 2024-04-25T11:20:14.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 shutdown_connections 2024-04-25T11:20:14.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa20c0b25e0 0x7fa20c1f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa20c1454a0 0x7fa20c0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa1ec06f400 0x7fa1ec071910 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 --2- 172.21.15.106:0/470150036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa20c1f9900 0x7fa20c1486f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 >> 172.21.15.106:0/470150036 conn(0x7fa20c06c7c0 msgr2=0x7fa20c14ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 shutdown_connections 2024-04-25T11:20:14.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.023+0000 7fa21087b640 1 -- 172.21.15.106:0/470150036 wait complete. 2024-04-25T11:20:14.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:14.042 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:14.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f9721b94640 1 Processor -- start 2024-04-25T11:20:14.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f9721b94640 1 -- start start 2024-04-25T11:20:14.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f9721b94640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f971c0cd930 0x7f971c0d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f9721b94640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971c0cdd40 con 0x7f971c0d1c80 2024-04-25T11:20:14.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f9721b94640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971c0d2090 con 0x7f971c143fa0 2024-04-25T11:20:14.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f9721b94640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f971c073300 con 0x7f971c0cd930 2024-04-25T11:20:14.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f971c0d1c80 0x7f971c0b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37368/0 (socket says 172.21.15.106:37368) 2024-04-25T11:20:14.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971bfff640 1 -- 172.21.15.106:0/869469030 learned_addr learned my addr 172.21.15.106:0/869469030 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:14.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971affd640 1 --2- 172.21.15.106:0/869469030 >> v2:172.21.15.134:3300/0 conn(0x7f971c0cd930 0x7f971c0d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3324332945 0 0) 0x7f97140014a0 con 0x7f971c0d1c80 2024-04-25T11:20:14.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9708003530 con 0x7f971c0d1c80 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1476307281 0 0) 0x7f9708003530 con 0x7f971c0d1c80 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 >> v2:172.21.15.134:3300/0 conn(0x7f971c0cd930 msgr2=0x7f971c0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 --2- 172.21.15.106:0/869469030 >> v2:172.21.15.134:3300/0 conn(0x7f971c0cd930 0x7f971c0d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 >> v1:172.21.15.134:6789/0 conn(0x7f971c143fa0 legacy=0x7f971c072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c0735e0 con 0x7f971c0d1c80 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f9714001bd0 con 0x7f971c0d1c80 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 >> v1:172.21.15.106:6789/0 conn(0x7f971c0d1c80 legacy=0x7f971c0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97080039c0 0x7f9708003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9708004810 0x7f9708004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97080051d0 0x7f970800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f971c073300 con 0x7f97080051d0 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f970800c170 con 0x7f9708004810 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f970800c370 con 0x7f97080039c0 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.195+0000 7f971bfff640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97080051d0 0x7f970800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971affd640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9708004810 0x7f9708004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971b7fe640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97080039c0 0x7f9708003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971bfff640 1 -- 172.21.15.106:0/869469030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97080039c0 msgr2=0x7f9708003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971bfff640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97080039c0 0x7f9708003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971bfff640 1 -- 172.21.15.106:0/869469030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9708004810 msgr2=0x7f9708004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971bfff640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9708004810 0x7f9708004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971bfff640 1 -- 172.21.15.106:0/869469030 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708003530 con 0x7f97080051d0 2024-04-25T11:20:14.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971bfff640 1 -- 172.21.15.106:0/869469030 reap_dead start 2024-04-25T11:20:14.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971bfff640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97080051d0 0x7f970800b9d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f971400aa00 tx=0x7f971401d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a288c24ef9d426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971400a650 con 0x7f97080051d0 2024-04-25T11:20:14.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97140ad650 con 0x7f97080051d0 2024-04-25T11:20:14.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f971a7fc640 1 -- 172.21.15.106:0/869469030 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140a5e60 con 0x7f97080051d0 2024-04-25T11:20:14.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f9721b94640 1 -- 172.21.15.106:0/869469030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97080051d0 msgr2=0x7f970800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f9721b94640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97080051d0 0x7f970800b9d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f971400aa00 tx=0x7f971401d280 comp rx=0 tx=0).stop 2024-04-25T11:20:14.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f9721b94640 1 -- 172.21.15.106:0/869469030 shutdown_connections 2024-04-25T11:20:14.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f9721b94640 1 --2- 172.21.15.106:0/869469030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97080051d0 0x7f970800b9d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f9721b94640 1 -- 172.21.15.106:0/869469030 >> 172.21.15.106:0/869469030 conn(0x7f971c06c9f0 msgr2=0x7f971c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f9721b94640 1 -- 172.21.15.106:0/869469030 shutdown_connections 2024-04-25T11:20:14.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.199+0000 7f9721b94640 1 -- 172.21.15.106:0/869469030 wait complete. 2024-04-25T11:20:14.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 Processor -- start 2024-04-25T11:20:14.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 -- start start 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97080051d0 0x7f971c13fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f971c0706f0 0x7f971c145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c145a70 0x7f971c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0b5320 con 0x7f971c0706f0 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0b5040 con 0x7f97080051d0 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f971c0d1740 con 0x7f971c145a70 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f971c0706f0 0x7f971c145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f971c0706f0 0x7f971c145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42160/0 (socket says 172.21.15.106:42160) 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 -- 172.21.15.106:0/2711231949 learned_addr learned my addr 172.21.15.106:0/2711231949 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:14.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971bfff640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c145a70 0x7f971c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 -- 172.21.15.106:0/2711231949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c145a70 msgr2=0x7f971c06ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c145a70 0x7f971c06ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971b7fe640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97080051d0 0x7f971c13fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 -- 172.21.15.106:0/2711231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97080051d0 msgr2=0x7f971c13fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97080051d0 0x7f971c13fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 -- 172.21.15.106:0/2711231949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c145e80 con 0x7f971c0706f0 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f971affd640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f971c0706f0 0x7f971c145530 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9704011590 tx=0x7f9704011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9713fff640 1 -- 172.21.15.106:0/2711231949 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704015870 con 0x7f971c0706f0 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9713fff640 1 -- 172.21.15.106:0/2711231949 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9704015a10 con 0x7f971c0706f0 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c1fec20 con 0x7f971c0706f0 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9713fff640 1 -- 172.21.15.106:0/2711231949 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704015dc0 con 0x7f971c0706f0 2024-04-25T11:20:14.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.203+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c1fde60 con 0x7f971c0706f0 2024-04-25T11:20:14.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.207+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f971c050750 con 0x7f971c0706f0 2024-04-25T11:20:14.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.207+0000 7f9713fff640 1 -- 172.21.15.106:0/2711231949 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9704013400 con 0x7f971c0706f0 2024-04-25T11:20:14.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.207+0000 7f9713fff640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f96fc06f400 0x7f96fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.207+0000 7f9713fff640 1 -- 172.21.15.106:0/2711231949 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f970408fad0 con 0x7f971c0706f0 2024-04-25T11:20:14.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.207+0000 7f971b7fe640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f96fc06f400 0x7f96fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.207+0000 7f971b7fe640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f96fc06f400 0x7f96fc071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f970c0014e0 tx=0x7f970c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.211+0000 7f9713fff640 1 -- 172.21.15.106:0/2711231949 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f971c145e80 con 0x7f971c0706f0 2024-04-25T11:20:14.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.375+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f971c15ebb0 con 0x7f971c0706f0 2024-04-25T11:20:14.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.375+0000 7f9713fff640 1 -- 172.21.15.106:0/2711231949 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f9704019030 con 0x7f971c0706f0 2024-04-25T11:20:14.376 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:14.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.375+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f96fc06f400 msgr2=0x7f96fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.375+0000 7f9721b94640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f96fc06f400 0x7f96fc071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f970c0014e0 tx=0x7f970c009040 comp rx=0 tx=0).stop 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.375+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f971c0706f0 msgr2=0x7f971c145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.375+0000 7f9721b94640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f971c0706f0 0x7f971c145530 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9704011590 tx=0x7f9704011a50 comp rx=0 tx=0).stop 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.379+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 shutdown_connections 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.379+0000 7f9721b94640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c145a70 0x7f971c06ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.379+0000 7f9721b94640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f96fc06f400 0x7f96fc071910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.379+0000 7f9721b94640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f971c0706f0 0x7f971c145530 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.379+0000 7f9721b94640 1 --2- 172.21.15.106:0/2711231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97080051d0 0x7f971c13fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.379+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 >> 172.21.15.106:0/2711231949 conn(0x7f971c06c9f0 msgr2=0x7f971c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.379+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 shutdown_connections 2024-04-25T11:20:14.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.379+0000 7f9721b94640 1 -- 172.21.15.106:0/2711231949 wait complete. 2024-04-25T11:20:14.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cbe50d640 1 Processor -- start 2024-04-25T11:20:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cbe50d640 1 -- start start 2024-04-25T11:20:14.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cbe50d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 0x7f1cb80b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cbe50d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb80d1350 con 0x7f1cb80d2150 2024-04-25T11:20:14.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cbe50d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb80b9270 con 0x7f1cb80d44a0 2024-04-25T11:20:14.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cbe50d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb8163cc0 con 0x7f1cb815fd10 2024-04-25T11:20:14.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb77fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1cb80d2150 0x7f1cb80d2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37374/0 (socket says 172.21.15.106:37374) 2024-04-25T11:20:14.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb77fe640 1 -- 172.21.15.106:0/3561270830 learned_addr learned my addr 172.21.15.106:0/3561270830 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:14.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb7fff640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 0x7f1cb80b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 709003494 0 0) 0x7f1cb80d1350 con 0x7f1cb80d2150 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ca0003530 con 0x7f1cb80d2150 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3095974007 0 0) 0x7f1ca0003530 con 0x7f1cb80d2150 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 msgr2=0x7f1cb80b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 0x7f1cb80b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 >> v1:172.21.15.134:6789/0 conn(0x7f1cb80d44a0 legacy=0x7f1cb80d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb8163fa0 con 0x7f1cb80d2150 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f1cac001680 con 0x7f1cb80d2150 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 >> v1:172.21.15.106:6789/0 conn(0x7f1cb80d2150 legacy=0x7f1cb80d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 0x7f1ca00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca0003f70 0x7f1ca0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0004d50 0x7f1ca0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8163cc0 con 0x7f1ca0003f70 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca000bb60 con 0x7f1ca0004d50 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca000bd60 con 0x7f1cb815fd10 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb77fe640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca0003f70 0x7f1ca0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb7fff640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 0x7f1ca00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cbcaaa640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0004d50 0x7f1ca0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb77fe640 1 -- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 msgr2=0x7f1ca00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb77fe640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 0x7f1ca00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb77fe640 1 -- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0004d50 msgr2=0x7f1ca0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb77fe640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0004d50 0x7f1ca0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb77fe640 1 -- 172.21.15.106:0/3561270830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca0003530 con 0x7f1ca0003f70 2024-04-25T11:20:14.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cb7fff640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb815fd10 0x7f1ca00039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:14.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.551+0000 7f1cbcaaa640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0004d50 0x7f1ca0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cb77fe640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca0003f70 0x7f1ca0004810 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c1c0 tx=0x7f1cac01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153aeb22495af6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac007d00 con 0x7f1ca0003f70 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cac005b00 con 0x7f1ca0003f70 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cb6ffd640 1 -- 172.21.15.106:0/3561270830 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac007070 con 0x7f1ca0003f70 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3561270830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca0003f70 msgr2=0x7f1ca0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cbe50d640 1 --2- 172.21.15.106:0/3561270830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca0003f70 0x7f1ca0004810 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c1c0 tx=0x7f1cac01f3d0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cbcaaa640 1 -- 172.21.15.106:0/3561270830 reap_dead start 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3561270830 shutdown_connections 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3561270830 >> 172.21.15.106:0/3561270830 conn(0x7f1cb806d7f0 msgr2=0x7f1cb806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3561270830 shutdown_connections 2024-04-25T11:20:14.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.555+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3561270830 wait complete. 2024-04-25T11:20:14.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 Processor -- start 2024-04-25T11:20:14.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 -- start start 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0003f70 0x7f1cb8163140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb81557b0 0x7f1cb8157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cb8158320 0x7f1cb81569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8160120 con 0x7f1cb8158320 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8164560 con 0x7f1ca0003f70 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb81638e0 con 0x7f1cb81557b0 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cb8158320 0x7f1cb81569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cb8158320 0x7f1cb81569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42176/0 (socket says 172.21.15.106:42176) 2024-04-25T11:20:14.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 -- 172.21.15.106:0/3466607389 learned_addr learned my addr 172.21.15.106:0/3466607389 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cb7fff640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0003f70 0x7f1cb8163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cb77fe640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb81557b0 0x7f1cb8157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 -- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb81557b0 msgr2=0x7f1cb8157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb81557b0 0x7f1cb8157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 -- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0003f70 msgr2=0x7f1cb8163140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0003f70 0x7f1cb8163140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 -- 172.21.15.106:0/3466607389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb8158730 con 0x7f1cb8158320 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cb7fff640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0003f70 0x7f1cb8163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbcaaa640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cb8158320 0x7f1cb81569a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1cb808dcd0 tx=0x7f1cb000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cb4ff9640 1 -- 172.21.15.106:0/3466607389 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0006d60 con 0x7f1cb8158320 2024-04-25T11:20:14.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cb4ff9640 1 -- 172.21.15.106:0/3466607389 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb0003980 con 0x7f1cb8158320 2024-04-25T11:20:14.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb8159ad0 con 0x7f1cb8158320 2024-04-25T11:20:14.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cb4ff9640 1 -- 172.21.15.106:0/3466607389 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0003db0 con 0x7f1cb8158320 2024-04-25T11:20:14.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.559+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb81f9d50 con 0x7f1cb8158320 2024-04-25T11:20:14.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.563+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb8050750 con 0x7f1cb8158320 2024-04-25T11:20:14.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.563+0000 7f1cb4ff9640 1 -- 172.21.15.106:0/3466607389 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1cb001b020 con 0x7f1cb8158320 2024-04-25T11:20:14.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.563+0000 7f1cb4ff9640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1ca406f430 0x7f1ca4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.563+0000 7f1cb7fff640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1ca406f430 0x7f1ca4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.563+0000 7f1cb4ff9640 1 -- 172.21.15.106:0/3466607389 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1cb008eed0 con 0x7f1cb8158320 2024-04-25T11:20:14.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.563+0000 7f1cb7fff640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1ca406f430 0x7f1ca4071940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009ac0 tx=0x7f1ca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.567+0000 7f1cb4ff9640 1 -- 172.21.15.106:0/3466607389 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1cb8158730 con 0x7f1cb8158320 2024-04-25T11:20:14.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.727+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1cb80d3be0 con 0x7f1cb8158320 2024-04-25T11:20:14.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cb4ff9640 1 -- 172.21.15.106:0/3466607389 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f1cb005bd40 con 0x7f1cb8158320 2024-04-25T11:20:14.730 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1ca406f430 msgr2=0x7f1ca4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1ca406f430 0x7f1ca4071940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009ac0 tx=0x7f1ca8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cb8158320 msgr2=0x7f1cb81569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cb8158320 0x7f1cb81569a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1cb808dcd0 tx=0x7f1cb000dc00 comp rx=0 tx=0).stop 2024-04-25T11:20:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 shutdown_connections 2024-04-25T11:20:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1ca406f430 0x7f1ca4071940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cb8158320 0x7f1cb81569a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cb81557b0 0x7f1cb8157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 --2- 172.21.15.106:0/3466607389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca0003f70 0x7f1cb8163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 >> 172.21.15.106:0/3466607389 conn(0x7f1cb806d7f0 msgr2=0x7f1cb80b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 shutdown_connections 2024-04-25T11:20:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.731+0000 7f1cbe50d640 1 -- 172.21.15.106:0/3466607389 wait complete. 2024-04-25T11:20:14.753 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:14.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.907+0000 7f271f5be640 1 Processor -- start 2024-04-25T11:20:14.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.907+0000 7f271f5be640 1 -- start start 2024-04-25T11:20:14.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.907+0000 7f271f5be640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2718073130 0x7f2718167000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.907+0000 7f271f5be640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27180b43f0 con 0x7f27180b6120 2024-04-25T11:20:14.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.907+0000 7f271f5be640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27180b46d0 con 0x7f27180706c0 2024-04-25T11:20:14.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.907+0000 7f271f5be640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f27180b6530 con 0x7f2718073130 2024-04-25T11:20:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271db5b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2718073130 0x7f2718167000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271d35a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f27180b6120 0x7f271815bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37390/0 (socket says 172.21.15.106:37390) 2024-04-25T11:20:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271d35a640 1 -- 172.21.15.106:0/1371721037 learned_addr learned my addr 172.21.15.106:0/1371721037 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271db5b640 1 -- 172.21.15.106:0/1371721037 >> v1:172.21.15.134:6789/0 conn(0x7f27180706c0 legacy=0x7f2718072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271db5b640 1 -- 172.21.15.106:0/1371721037 >> v1:172.21.15.106:6789/0 conn(0x7f27180b6120 legacy=0x7f271815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f270ffff640 1 -- 172.21.15.106:0/1371721037 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160756027 0 0) 0x7f27180b43f0 con 0x7f27180b6120 2024-04-25T11:20:14.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271db5b640 1 -- 172.21.15.106:0/1371721037 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27180b49b0 con 0x7f2718073130 2024-04-25T11:20:14.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271db5b640 1 --2- 172.21.15.106:0/1371721037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2718073130 0x7f2718167000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2710009d90 tx=0x7f271001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5080218ad66d13c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f270ffff640 1 -- 172.21.15.106:0/1371721037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27100b2070 con 0x7f2718073130 2024-04-25T11:20:14.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f270ffff640 1 -- 172.21.15.106:0/1371721037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27100a5880 con 0x7f2718073130 2024-04-25T11:20:14.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f270ffff640 1 -- 172.21.15.106:0/1371721037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27100a5c50 con 0x7f2718073130 2024-04-25T11:20:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271f5be640 1 -- 172.21.15.106:0/1371721037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2718073130 msgr2=0x7f2718167000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271f5be640 1 --2- 172.21.15.106:0/1371721037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2718073130 0x7f2718167000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2710009d90 tx=0x7f271001c930 comp rx=0 tx=0).stop 2024-04-25T11:20:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271f5be640 1 -- 172.21.15.106:0/1371721037 shutdown_connections 2024-04-25T11:20:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271f5be640 1 --2- 172.21.15.106:0/1371721037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2718073130 0x7f2718167000 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271f5be640 1 -- 172.21.15.106:0/1371721037 >> 172.21.15.106:0/1371721037 conn(0x7f271806c9f0 msgr2=0x7f271806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271f5be640 1 -- 172.21.15.106:0/1371721037 shutdown_connections 2024-04-25T11:20:14.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.911+0000 7f271f5be640 1 -- 172.21.15.106:0/1371721037 wait complete. 2024-04-25T11:20:14.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271f5be640 1 Processor -- start 2024-04-25T11:20:14.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271f5be640 1 -- start start 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271f5be640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27180706c0 0x7f2718140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271f5be640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27180b6120 0x7f271814ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271f5be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2718150500 0x7f2718149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271f5be640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f271816a880 con 0x7f27180706c0 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271f5be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f271816aa00 con 0x7f2718150500 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271f5be640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f271816ab80 con 0x7f27180b6120 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27180706c0 0x7f2718140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27180706c0 0x7f2718140420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42192/0 (socket says 172.21.15.106:42192) 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 -- 172.21.15.106:0/839961180 learned_addr learned my addr 172.21.15.106:0/839961180 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271db5b640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2718150500 0x7f2718149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271cb59640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27180b6120 0x7f271814ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 -- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27180b6120 msgr2=0x7f271814ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27180b6120 0x7f271814ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 -- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2718150500 msgr2=0x7f2718149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2718150500 0x7f2718149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 -- 172.21.15.106:0/839961180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2718147dc0 con 0x7f27180706c0 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271cb59640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27180b6120 0x7f271814ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f271d35a640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27180706c0 0x7f2718140420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f270800c4b0 tx=0x7f270800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f270dffb640 1 -- 172.21.15.106:0/839961180 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708018070 con 0x7f27180706c0 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f270dffb640 1 -- 172.21.15.106:0/839961180 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2708007500 con 0x7f27180706c0 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.915+0000 7f270dffb640 1 -- 172.21.15.106:0/839961180 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708004a30 con 0x7f27180706c0 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.919+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27181480a0 con 0x7f27180706c0 2024-04-25T11:20:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.919+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2718148660 con 0x7f27180706c0 2024-04-25T11:20:14.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.919+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271806fec0 con 0x7f27180706c0 2024-04-25T11:20:14.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.919+0000 7f270dffb640 1 -- 172.21.15.106:0/839961180 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2708004bd0 con 0x7f27180706c0 2024-04-25T11:20:14.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.919+0000 7f270dffb640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f26fc06f400 0x7f26fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.919+0000 7f270dffb640 1 -- 172.21.15.106:0/839961180 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f270808e5d0 con 0x7f27180706c0 2024-04-25T11:20:14.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.919+0000 7f271cb59640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f26fc06f400 0x7f26fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.919+0000 7f271cb59640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f26fc06f400 0x7f26fc071910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2718151a40 tx=0x7f2714008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:14.923+0000 7f270dffb640 1 -- 172.21.15.106:0/839961180 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f270805b440 con 0x7f27180706c0 2024-04-25T11:20:15.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.087+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f271815b790 con 0x7f27180706c0 2024-04-25T11:20:15.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.087+0000 7f270dffb640 1 -- 172.21.15.106:0/839961180 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f270805ad40 con 0x7f27180706c0 2024-04-25T11:20:15.088 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f26fc06f400 msgr2=0x7f26fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f26fc06f400 0x7f26fc071910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2718151a40 tx=0x7f2714008040 comp rx=0 tx=0).stop 2024-04-25T11:20:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27180706c0 msgr2=0x7f2718140420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27180706c0 0x7f2718140420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f270800c4b0 tx=0x7f270800c970 comp rx=0 tx=0).stop 2024-04-25T11:20:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 shutdown_connections 2024-04-25T11:20:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2718150500 0x7f2718149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27180b6120 0x7f271814ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f26fc06f400 0x7f26fc071910 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 --2- 172.21.15.106:0/839961180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27180706c0 0x7f2718140420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 >> 172.21.15.106:0/839961180 conn(0x7f271806c9f0 msgr2=0x7f2718159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 shutdown_connections 2024-04-25T11:20:15.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.091+0000 7f271f5be640 1 -- 172.21.15.106:0/839961180 wait complete. 2024-04-25T11:20:15.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:15.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.247+0000 7fb2f9be3640 1 Processor -- start 2024-04-25T11:20:15.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.247+0000 7fb2f9be3640 1 -- start start 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f9be3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb2f40b88a0 0x7fb2f40b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f9be3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 0x7fb2f409be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f9be3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2f40d17a0 0x7fb2f40d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f9be3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb2f40b2ce0 con 0x7fb2f40b88a0 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f9be3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb2f409cf60 con 0x7fb2f40d17a0 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f9be3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f409c380 con 0x7fb2f409ba30 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb2f40b88a0 0x7fb2f40b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb2f40b88a0 0x7fb2f40b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42208/0 (socket says 172.21.15.106:42208) 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 -- 172.21.15.106:0/1105540116 learned_addr learned my addr 172.21.15.106:0/1105540116 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f2ffd640 1 --2- 172.21.15.106:0/1105540116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 0x7fb2f409be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 -- 172.21.15.106:0/1105540116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 msgr2=0x7fb2f409be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f3fff640 1 --2- 172.21.15.106:0/1105540116 >> v2:172.21.15.134:3300/0 conn(0x7fb2f40d17a0 0x7fb2f40d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 --2- 172.21.15.106:0/1105540116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 0x7fb2f409be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 -- 172.21.15.106:0/1105540116 >> v2:172.21.15.134:3300/0 conn(0x7fb2f40d17a0 msgr2=0x7fb2f40d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:15.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 --2- 172.21.15.106:0/1105540116 >> v2:172.21.15.134:3300/0 conn(0x7fb2f40d17a0 0x7fb2f40d1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 -- 172.21.15.106:0/1105540116 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f40cef70 con 0x7fb2f40b88a0 2024-04-25T11:20:15.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f37fe640 1 --2- 172.21.15.106:0/1105540116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40b88a0 0x7fb2f40b2640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4076830 tx=0x7fb2ec01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e8502ff43215da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f27fc640 1 -- 172.21.15.106:0/1105540116 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0a57f0 con 0x7fb2f40b88a0 2024-04-25T11:20:15.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f27fc640 1 -- 172.21.15.106:0/1105540116 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ec0a5990 con 0x7fb2f40b88a0 2024-04-25T11:20:15.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.251+0000 7fb2f27fc640 1 -- 172.21.15.106:0/1105540116 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0a5d60 con 0x7fb2f40b88a0 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- 172.21.15.106:0/1105540116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40b88a0 msgr2=0x7fb2f40b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/1105540116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40b88a0 0x7fb2f40b2640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4076830 tx=0x7fb2ec01ca50 comp rx=0 tx=0).stop 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- 172.21.15.106:0/1105540116 shutdown_connections 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/1105540116 >> v2:172.21.15.134:3300/0 conn(0x7fb2f40d17a0 0x7fb2f40d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/1105540116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 0x7fb2f409be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/1105540116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40b88a0 0x7fb2f40b2640 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- 172.21.15.106:0/1105540116 >> 172.21.15.106:0/1105540116 conn(0x7fb2f40b5060 msgr2=0x7fb2f40b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- 172.21.15.106:0/1105540116 shutdown_connections 2024-04-25T11:20:15.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- 172.21.15.106:0/1105540116 wait complete. 2024-04-25T11:20:15.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 Processor -- start 2024-04-25T11:20:15.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- start start 2024-04-25T11:20:15.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 0x7fb2f40c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f40b88a0 0x7fb2f415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40d17a0 0x7fb2f415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f40b3980 con 0x7fb2f40d17a0 2024-04-25T11:20:15.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f40d2500 con 0x7fb2f40b88a0 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f9be3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f40b3b00 con 0x7fb2f409ba30 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40d17a0 0x7fb2f415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40d17a0 0x7fb2f415b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42214/0 (socket says 172.21.15.106:42214) 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f3fff640 1 -- 172.21.15.106:0/463071727 learned_addr learned my addr 172.21.15.106:0/463071727 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f37fe640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 0x7fb2f40c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f2ffd640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f40b88a0 0x7fb2f415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f3fff640 1 -- 172.21.15.106:0/463071727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 msgr2=0x7fb2f40c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f3fff640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 0x7fb2f40c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f3fff640 1 -- 172.21.15.106:0/463071727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f40b88a0 msgr2=0x7fb2f415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f3fff640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f40b88a0 0x7fb2f415ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.255+0000 7fb2f3fff640 1 -- 172.21.15.106:0/463071727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f40c7780 con 0x7fb2f40d17a0 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2f3fff640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40d17a0 0x7fb2f415b0d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c540 tx=0x7fb2e400ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2dbfff640 1 -- 172.21.15.106:0/463071727 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4015070 con 0x7fb2f40d17a0 2024-04-25T11:20:15.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f40c7a60 con 0x7fb2f40d17a0 2024-04-25T11:20:15.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2dbfff640 1 -- 172.21.15.106:0/463071727 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e4004220 con 0x7fb2f40d17a0 2024-04-25T11:20:15.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2dbfff640 1 -- 172.21.15.106:0/463071727 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4003620 con 0x7fb2f40d17a0 2024-04-25T11:20:15.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f40c8020 con 0x7fb2f40d17a0 2024-04-25T11:20:15.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f4050750 con 0x7fb2f40d17a0 2024-04-25T11:20:15.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2dbfff640 1 -- 172.21.15.106:0/463071727 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb2e40037c0 con 0x7fb2f40d17a0 2024-04-25T11:20:15.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2dbfff640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2e006f400 0x7fb2e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2dbfff640 1 -- 172.21.15.106:0/463071727 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb2e408f0a0 con 0x7fb2f40d17a0 2024-04-25T11:20:15.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2f37fe640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2e006f400 0x7fb2e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.259+0000 7fb2f37fe640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2e006f400 0x7fb2e0071910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0ae000 tx=0x7fb2ec0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.263+0000 7fb2dbfff640 1 -- 172.21.15.106:0/463071727 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2e405bf20 con 0x7fb2f40d17a0 2024-04-25T11:20:15.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb2f40ced20 con 0x7fb2f40d17a0 2024-04-25T11:20:15.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2dbfff640 1 -- 172.21.15.106:0/463071727 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fb2e405b820 con 0x7fb2f40d17a0 2024-04-25T11:20:15.442 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2e006f400 msgr2=0x7fb2e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2e006f400 0x7fb2e0071910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0ae000 tx=0x7fb2ec0a7070 comp rx=0 tx=0).stop 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40d17a0 msgr2=0x7fb2f415b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40d17a0 0x7fb2f415b0d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c540 tx=0x7fb2e400ca00 comp rx=0 tx=0).stop 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 shutdown_connections 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2e006f400 0x7fb2e0071910 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f40d17a0 0x7fb2f415b0d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f40b88a0 0x7fb2f415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 --2- 172.21.15.106:0/463071727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f409ba30 0x7fb2f40c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 >> 172.21.15.106:0/463071727 conn(0x7fb2f40b5060 msgr2=0x7fb2f40b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 shutdown_connections 2024-04-25T11:20:15.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.443+0000 7fb2f9be3640 1 -- 172.21.15.106:0/463071727 wait complete. 2024-04-25T11:20:15.463 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:20:15.464 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:15.470 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:15.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd945d89640 1 Processor -- start 2024-04-25T11:20:15.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd945d89640 1 -- start start 2024-04-25T11:20:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd945d89640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd9400cecf0 0x7fd940164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd945d89640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd9400b1dd0 0x7fd9400b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd945d89640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9400b4d60 con 0x7fd9400c5440 2024-04-25T11:20:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd945d89640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd9400d0f30 con 0x7fd9400cecf0 2024-04-25T11:20:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd945d89640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd9400d1210 con 0x7fd9400b1dd0 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd9400cecf0 0x7fd940164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd9400c5440 0x7fd9400c5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49024/0 (socket says 172.21.15.106:49024) 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93effd640 1 -- 172.21.15.106:0/2125483978 learned_addr learned my addr 172.21.15.106:0/2125483978 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93ffff640 1 --2- 172.21.15.106:0/2125483978 >> v2:172.21.15.134:3300/0 conn(0x7fd9400b1dd0 0x7fd9400b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93f7fe640 1 -- 172.21.15.106:0/2125483978 >> v2:172.21.15.134:3300/0 conn(0x7fd9400b1dd0 msgr2=0x7fd9400b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93f7fe640 1 --2- 172.21.15.106:0/2125483978 >> v2:172.21.15.134:3300/0 conn(0x7fd9400b1dd0 0x7fd9400b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93f7fe640 1 -- 172.21.15.106:0/2125483978 >> v1:172.21.15.106:6789/0 conn(0x7fd9400c5440 legacy=0x7fd9400c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93e7fc640 1 -- 172.21.15.106:0/2125483978 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294677230 0 0) 0x7fd934001120 con 0x7fd9400c5440 2024-04-25T11:20:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.611+0000 7fd93f7fe640 1 -- 172.21.15.106:0/2125483978 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9400d14f0 con 0x7fd9400cecf0 2024-04-25T11:20:15.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd93f7fe640 1 --2- 172.21.15.106:0/2125483978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd940164310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd940074bc0 tx=0x7fd93801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad56ee4a9713c007 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd93e7fc640 1 -- 172.21.15.106:0/2125483978 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9380b2070 con 0x7fd9400cecf0 2024-04-25T11:20:15.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd93e7fc640 1 -- 172.21.15.106:0/2125483978 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9380a5870 con 0x7fd9400cecf0 2024-04-25T11:20:15.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd93e7fc640 1 -- 172.21.15.106:0/2125483978 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9380a5c40 con 0x7fd9400cecf0 2024-04-25T11:20:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd945d89640 1 -- 172.21.15.106:0/2125483978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 msgr2=0x7fd940164310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd945d89640 1 --2- 172.21.15.106:0/2125483978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd940164310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd940074bc0 tx=0x7fd93801c8a0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd945d89640 1 -- 172.21.15.106:0/2125483978 shutdown_connections 2024-04-25T11:20:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd945d89640 1 --2- 172.21.15.106:0/2125483978 >> v2:172.21.15.134:3300/0 conn(0x7fd9400b1dd0 0x7fd9400b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd945d89640 1 --2- 172.21.15.106:0/2125483978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd940164310 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd945d89640 1 -- 172.21.15.106:0/2125483978 >> 172.21.15.106:0/2125483978 conn(0x7fd9400b4020 msgr2=0x7fd9400b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd945d89640 1 -- 172.21.15.106:0/2125483978 shutdown_connections 2024-04-25T11:20:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.615+0000 7fd945d89640 1 -- 172.21.15.106:0/2125483978 wait complete. 2024-04-25T11:20:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 Processor -- start 2024-04-25T11:20:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 -- start start 2024-04-25T11:20:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9400b1dd0 0x7fd94014b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9400c5440 0x7fd9401492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd9401489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401711b0 con 0x7fd9400cecf0 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd940171030 con 0x7fd9400c5440 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd940171330 con 0x7fd9400b1dd0 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd9401489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd9401489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59164/0 (socket says 172.21.15.106:59164) 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 -- 172.21.15.106:0/2181602634 learned_addr learned my addr 172.21.15.106:0/2181602634 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 -- 172.21.15.106:0/2181602634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9400b1dd0 msgr2=0x7fd94014b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9400b1dd0 0x7fd94014b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 -- 172.21.15.106:0/2181602634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9400c5440 msgr2=0x7fd9401492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9400c5440 0x7fd9401492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 -- 172.21.15.106:0/2181602634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94014bd30 con 0x7fd9400cecf0 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd93ffff640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd9401489f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd93000d590 tx=0x7fd93000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd927fff640 1 -- 172.21.15.106:0/2181602634 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9300125f0 con 0x7fd9400cecf0 2024-04-25T11:20:15.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd927fff640 1 -- 172.21.15.106:0/2181602634 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd930019070 con 0x7fd9400cecf0 2024-04-25T11:20:15.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94014c010 con 0x7fd9400cecf0 2024-04-25T11:20:15.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd927fff640 1 -- 172.21.15.106:0/2181602634 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9300129f0 con 0x7fd9400cecf0 2024-04-25T11:20:15.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.619+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd940166d70 con 0x7fd9400cecf0 2024-04-25T11:20:15.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.623+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd940164630 con 0x7fd9400cecf0 2024-04-25T11:20:15.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.623+0000 7fd927fff640 1 -- 172.21.15.106:0/2181602634 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd930012b90 con 0x7fd9400cecf0 2024-04-25T11:20:15.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.623+0000 7fd927fff640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd92c06f400 0x7fd92c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.623+0000 7fd927fff640 1 -- 172.21.15.106:0/2181602634 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd93008f750 con 0x7fd9400cecf0 2024-04-25T11:20:15.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.623+0000 7fd93f7fe640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd92c06f400 0x7fd92c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.623+0000 7fd93f7fe640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd92c06f400 0x7fd92c071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9380af000 tx=0x7fd938004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.627+0000 7fd927fff640 1 -- 172.21.15.106:0/2181602634 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd93005c5c0 con 0x7fd9400cecf0 2024-04-25T11:20:15.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.787+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd94014bac0 con 0x7fd9400cecf0 2024-04-25T11:20:15.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd927fff640 1 -- 172.21.15.106:0/2181602634 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16834 (secure 0 0 0) 0x7fd93005bec0 con 0x7fd9400cecf0 2024-04-25T11:20:15.790 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:15.790 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:07.237284+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:15.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd92c06f400 msgr2=0x7fd92c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd92c06f400 0x7fd92c071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9380af000 tx=0x7fd938004ac0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 msgr2=0x7fd9401489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd9401489f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd93000d590 tx=0x7fd93000da50 comp rx=0 tx=0).stop 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 shutdown_connections 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd92c06f400 0x7fd92c071910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9400cecf0 0x7fd9401489f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9400c5440 0x7fd9401492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 --2- 172.21.15.106:0/2181602634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9400b1dd0 0x7fd94014b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 >> 172.21.15.106:0/2181602634 conn(0x7fd9400b4020 msgr2=0x7fd9400af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 shutdown_connections 2024-04-25T11:20:15.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.791+0000 7fd945d89640 1 -- 172.21.15.106:0/2181602634 wait complete. 2024-04-25T11:20:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:15.812 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:15.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f958b341640 1 Processor -- start 2024-04-25T11:20:15.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f958b341640 1 -- start start 2024-04-25T11:20:15.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f958b341640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f95840af400 0x7f95840af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f958b341640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f958415f310 0x7f958415f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f958b341640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95840b4ca0 con 0x7f958406c5f0 2024-04-25T11:20:15.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f958b341640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f958415f160 con 0x7f958415f310 2024-04-25T11:20:15.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f958b341640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f95840b4f80 con 0x7f95840af400 2024-04-25T11:20:15.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95898de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f958415f310 0x7f958415f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95888dc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f958406c5f0 0x7f958406ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49036/0 (socket says 172.21.15.106:49036) 2024-04-25T11:20:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95888dc640 1 -- 172.21.15.106:0/838662405 learned_addr learned my addr 172.21.15.106:0/838662405 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95890dd640 1 --2- 172.21.15.106:0/838662405 >> v2:172.21.15.134:3300/0 conn(0x7f95840af400 0x7f95840af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95898de640 1 -- 172.21.15.106:0/838662405 >> v2:172.21.15.134:3300/0 conn(0x7f95840af400 msgr2=0x7f95840af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95898de640 1 --2- 172.21.15.106:0/838662405 >> v2:172.21.15.134:3300/0 conn(0x7f95840af400 0x7f95840af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f957bfff640 1 -- 172.21.15.106:0/838662405 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 274425390 0 0) 0x7f95840b4ca0 con 0x7f958406c5f0 2024-04-25T11:20:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95898de640 1 -- 172.21.15.106:0/838662405 >> v1:172.21.15.106:6789/0 conn(0x7f958406c5f0 legacy=0x7f958406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95898de640 1 -- 172.21.15.106:0/838662405 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958415e260 con 0x7f958415f310 2024-04-25T11:20:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.951+0000 7f95890dd640 1 --2- 172.21.15.106:0/838662405 >> v2:172.21.15.134:3300/0 conn(0x7f95840af400 0x7f95840af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f95898de640 1 --2- 172.21.15.106:0/838662405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f958415f720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9584073300 tx=0x7f958001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7366a5b7627a8c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f957bfff640 1 -- 172.21.15.106:0/838662405 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800b2070 con 0x7f958415f310 2024-04-25T11:20:15.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f957bfff640 1 -- 172.21.15.106:0/838662405 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95800a58d0 con 0x7f958415f310 2024-04-25T11:20:15.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f957bfff640 1 -- 172.21.15.106:0/838662405 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800a5ca0 con 0x7f958415f310 2024-04-25T11:20:15.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f958b341640 1 -- 172.21.15.106:0/838662405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 msgr2=0x7f958415f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f958b341640 1 --2- 172.21.15.106:0/838662405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f958415f720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9584073300 tx=0x7f958001c900 comp rx=0 tx=0).stop 2024-04-25T11:20:15.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f958b341640 1 -- 172.21.15.106:0/838662405 shutdown_connections 2024-04-25T11:20:15.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f958b341640 1 --2- 172.21.15.106:0/838662405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f958415f720 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f958b341640 1 --2- 172.21.15.106:0/838662405 >> v2:172.21.15.134:3300/0 conn(0x7f95840af400 0x7f95840af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f958b341640 1 -- 172.21.15.106:0/838662405 >> 172.21.15.106:0/838662405 conn(0x7f958409ba30 msgr2=0x7f95840b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f958b341640 1 -- 172.21.15.106:0/838662405 shutdown_connections 2024-04-25T11:20:15.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.955+0000 7f958b341640 1 -- 172.21.15.106:0/838662405 wait complete. 2024-04-25T11:20:15.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 Processor -- start 2024-04-25T11:20:15.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 -- start start 2024-04-25T11:20:15.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f958406c5f0 0x7f9584156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95840af400 0x7f9584151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f9584154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f95840b4830 con 0x7f958415f310 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f958415bbf0 con 0x7f958406c5f0 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95840b49b0 con 0x7f95840af400 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f9584154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f9584154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59182/0 (socket says 172.21.15.106:59182) 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 -- 172.21.15.106:0/2548704949 learned_addr learned my addr 172.21.15.106:0/2548704949 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 -- 172.21.15.106:0/2548704949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95840af400 msgr2=0x7f9584151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95840af400 0x7f9584151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95890dd640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f958406c5f0 0x7f9584156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 -- 172.21.15.106:0/2548704949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f958406c5f0 msgr2=0x7f9584156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f958406c5f0 0x7f9584156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 -- 172.21.15.106:0/2548704949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9584156fe0 con 0x7f958415f310 2024-04-25T11:20:15.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f95898de640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f9584154a00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f958001a530 tx=0x7f95800196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f9579ffb640 1 -- 172.21.15.106:0/2548704949 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800b2050 con 0x7f958415f310 2024-04-25T11:20:15.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f9579ffb640 1 -- 172.21.15.106:0/2548704949 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95800be070 con 0x7f958415f310 2024-04-25T11:20:15.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95841572c0 con 0x7f958415f310 2024-04-25T11:20:15.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f9579ffb640 1 -- 172.21.15.106:0/2548704949 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800ada50 con 0x7f958415f310 2024-04-25T11:20:15.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9584156aa0 con 0x7f958415f310 2024-04-25T11:20:15.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.959+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9584156d50 con 0x7f958415f310 2024-04-25T11:20:15.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.963+0000 7f9579ffb640 1 -- 172.21.15.106:0/2548704949 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95800adbf0 con 0x7f958415f310 2024-04-25T11:20:15.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.963+0000 7f9579ffb640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f956806f430 0x7f9568071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.963+0000 7f9579ffb640 1 -- 172.21.15.106:0/2548704949 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9580129ff0 con 0x7f958415f310 2024-04-25T11:20:15.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.963+0000 7f95890dd640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f956806f430 0x7f9568071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.963+0000 7f95890dd640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f956806f430 0x7f9568071940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f957c006fd0 tx=0x7f957c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:15.967+0000 7f9579ffb640 1 -- 172.21.15.106:0/2548704949 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95800c7020 con 0x7f958415f310 2024-04-25T11:20:16.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9584050750 con 0x7f958415f310 2024-04-25T11:20:16.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f9579ffb640 1 -- 172.21.15.106:0/2548704949 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f9580016520 con 0x7f958415f310 2024-04-25T11:20:16.115 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:16.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f956806f430 msgr2=0x7f9568071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f956806f430 0x7f9568071940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f957c006fd0 tx=0x7f957c008040 comp rx=0 tx=0).stop 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 msgr2=0x7f9584154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f9584154a00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f958001a530 tx=0x7f95800196a0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 shutdown_connections 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f956806f430 0x7f9568071940 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f958415f310 0x7f9584154a00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95840af400 0x7f9584151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.115+0000 7f958b341640 1 --2- 172.21.15.106:0/2548704949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f958406c5f0 0x7f9584156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.119+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 >> 172.21.15.106:0/2548704949 conn(0x7f958409ba30 msgr2=0x7f9584153a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.119+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 shutdown_connections 2024-04-25T11:20:16.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.119+0000 7f958b341640 1 -- 172.21.15.106:0/2548704949 wait complete. 2024-04-25T11:20:16.135 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:16.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47ee9be640 1 Processor -- start 2024-04-25T11:20:16.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47ee9be640 1 -- start start 2024-04-25T11:20:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47ee9be640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f47e815fd10 0x7f47e80b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47ee9be640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d2150 0x7f47e80d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47ee9be640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47e80d1350 con 0x7f47e80d44a0 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47ee9be640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f47e80d2aa0 con 0x7f47e815fd10 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47ee9be640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f47e80b9270 con 0x7f47e80d2150 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47e7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f47e815fd10 0x7f47e80b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47e7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f47e815fd10 0x7f47e80b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59184/0 (socket says 172.21.15.106:59184) 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47e7fff640 1 -- 172.21.15.106:0/597080378 learned_addr learned my addr 172.21.15.106:0/597080378 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47dffff640 1 --2- 172.21.15.106:0/597080378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d2150 0x7f47e80d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47e7fff640 1 -- 172.21.15.106:0/597080378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d2150 msgr2=0x7f47e80d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47e7fff640 1 --2- 172.21.15.106:0/597080378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d2150 0x7f47e80d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47e7fff640 1 -- 172.21.15.106:0/597080378 >> v1:172.21.15.134:6789/0 conn(0x7f47e80d44a0 legacy=0x7f47e80d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.275+0000 7f47e7fff640 1 -- 172.21.15.106:0/597080378 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e8163cc0 con 0x7f47e815fd10 2024-04-25T11:20:16.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47e7fff640 1 --2- 172.21.15.106:0/597080378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80b8d30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009870 tx=0x7f47d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b035abe66690a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47e77fe640 1 -- 172.21.15.106:0/597080378 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80b1070 con 0x7f47e815fd10 2024-04-25T11:20:16.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47e77fe640 1 -- 172.21.15.106:0/597080378 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d80ac630 con 0x7f47e815fd10 2024-04-25T11:20:16.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47e77fe640 1 -- 172.21.15.106:0/597080378 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80a8050 con 0x7f47e815fd10 2024-04-25T11:20:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 -- 172.21.15.106:0/597080378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 msgr2=0x7f47e80b8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 --2- 172.21.15.106:0/597080378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80b8d30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009870 tx=0x7f47d801c520 comp rx=0 tx=0).stop 2024-04-25T11:20:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 -- 172.21.15.106:0/597080378 shutdown_connections 2024-04-25T11:20:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 --2- 172.21.15.106:0/597080378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d2150 0x7f47e80d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 --2- 172.21.15.106:0/597080378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80b8d30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 -- 172.21.15.106:0/597080378 >> 172.21.15.106:0/597080378 conn(0x7f47e806d7f0 msgr2=0x7f47e806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 -- 172.21.15.106:0/597080378 shutdown_connections 2024-04-25T11:20:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 -- 172.21.15.106:0/597080378 wait complete. 2024-04-25T11:20:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 Processor -- start 2024-04-25T11:20:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.279+0000 7f47ee9be640 1 -- start start 2024-04-25T11:20:16.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e80d2150 0x7f47e814ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d44a0 0x7f47e815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81638e0 con 0x7f47e815fd10 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8160120 con 0x7f47e80d2150 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f47e8163a60 con 0x7f47e80d44a0 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80ba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59190/0 (socket says 172.21.15.106:59190) 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 -- 172.21.15.106:0/2517712767 learned_addr learned my addr 172.21.15.106:0/2517712767 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47e7fff640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e80d2150 0x7f47e814ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47dffff640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d44a0 0x7f47e815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 -- 172.21.15.106:0/2517712767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d44a0 msgr2=0x7f47e815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d44a0 0x7f47e815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 -- 172.21.15.106:0/2517712767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e80d2150 msgr2=0x7f47e814ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e80d2150 0x7f47e814ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 -- 172.21.15.106:0/2517712767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e815e3b0 con 0x7f47e815fd10 2024-04-25T11:20:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ecf5b640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80ba6a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47e000ba10 tx=0x7f47e000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47e57fa640 1 -- 172.21.15.106:0/2517712767 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0019070 con 0x7f47e815fd10 2024-04-25T11:20:16.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47e57fa640 1 -- 172.21.15.106:0/2517712767 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e000cb40 con 0x7f47e815fd10 2024-04-25T11:20:16.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e815e690 con 0x7f47e815fd10 2024-04-25T11:20:16.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47e57fa640 1 -- 172.21.15.106:0/2517712767 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0014440 con 0x7f47e815fd10 2024-04-25T11:20:16.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e815ec50 con 0x7f47e815fd10 2024-04-25T11:20:16.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e8050750 con 0x7f47e815fd10 2024-04-25T11:20:16.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47e57fa640 1 -- 172.21.15.106:0/2517712767 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f47e001c070 con 0x7f47e815fd10 2024-04-25T11:20:16.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47e57fa640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f47c806f400 0x7f47c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47e57fa640 1 -- 172.21.15.106:0/2517712767 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f47e008efb0 con 0x7f47e815fd10 2024-04-25T11:20:16.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.283+0000 7f47e7fff640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f47c806f400 0x7f47c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.287+0000 7f47e7fff640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f47c806f400 0x7f47c8071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f47d80ae000 tx=0x7f47d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.291+0000 7f47e57fa640 1 -- 172.21.15.106:0/2517712767 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f47e005beb0 con 0x7f47e815fd10 2024-04-25T11:20:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.435+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47e80d4d70 con 0x7f47e815fd10 2024-04-25T11:20:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47e57fa640 1 -- 172.21.15.106:0/2517712767 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f47e005b7b0 con 0x7f47e815fd10 2024-04-25T11:20:16.438 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f47c806f400 msgr2=0x7f47c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f47c806f400 0x7f47c8071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f47d80ae000 tx=0x7f47d8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 msgr2=0x7f47e80ba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80ba6a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47e000ba10 tx=0x7f47e000bed0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 shutdown_connections 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f47c806f400 0x7f47c8071910 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47e815fd10 0x7f47e80ba6a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47e80d44a0 0x7f47e815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 --2- 172.21.15.106:0/2517712767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e80d2150 0x7f47e814ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 >> 172.21.15.106:0/2517712767 conn(0x7f47e806d7f0 msgr2=0x7f47e806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:16.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 shutdown_connections 2024-04-25T11:20:16.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.439+0000 7f47ee9be640 1 -- 172.21.15.106:0/2517712767 wait complete. 2024-04-25T11:20:16.458 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:16.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.611+0000 7fb2ff45e640 1 Processor -- start 2024-04-25T11:20:16.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.611+0000 7fb2ff45e640 1 -- start start 2024-04-25T11:20:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2ff45e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2f8143fa0 0x7fb2f80cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2ff45e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f8072a20 0x7fb2f80d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2ff45e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2f8073300 con 0x7fb2f8072e30 2024-04-25T11:20:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2ff45e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb2f80735e0 con 0x7fb2f8143fa0 2024-04-25T11:20:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2ff45e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f80738c0 con 0x7fb2f8072a20 2024-04-25T11:20:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd9fb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb2f8072e30 0x7fb2f80b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49040/0 (socket says 172.21.15.106:49040) 2024-04-25T11:20:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd9fb640 1 -- 172.21.15.106:0/440098147 learned_addr learned my addr 172.21.15.106:0/440098147 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd1fa640 1 --2- 172.21.15.106:0/440098147 >> v2:172.21.15.134:3300/0 conn(0x7fb2f8143fa0 0x7fb2f80cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fc9f9640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f8072a20 0x7fb2f80d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965176856 0 0) 0x7fb2f8073300 con 0x7fb2f8072e30 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2e4003530 con 0x7fb2f8072e30 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1495999868 0 0) 0x7fb2e4003530 con 0x7fb2f8072e30 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f8072a20 msgr2=0x7fb2f80d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f8072a20 0x7fb2f80d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 >> v2:172.21.15.134:3300/0 conn(0x7fb2f8143fa0 msgr2=0x7fb2f80cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 --2- 172.21.15.106:0/440098147 >> v2:172.21.15.134:3300/0 conn(0x7fb2f8143fa0 0x7fb2f80cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f80b5040 con 0x7fb2f8072e30 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fb2f0001a00 con 0x7fb2f8072e30 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 >> v1:172.21.15.106:6789/0 conn(0x7fb2f8072e30 legacy=0x7fb2f80b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e40039c0 0x7fb2e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f8072a20 0x7fb2e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2e4004d50 0x7fb2e4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f80735e0 con 0x7fb2e40039c0 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f80738c0 con 0x7fb2e4004d50 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e400b9d0 con 0x7fb2f8072a20 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd1fa640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e40039c0 0x7fb2e4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd9fb640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2e4004d50 0x7fb2e4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd1fa640 1 -- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f8072a20 msgr2=0x7fb2e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd1fa640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2f8072a20 0x7fb2e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd1fa640 1 -- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2e4004d50 msgr2=0x7fb2e4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd1fa640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2e4004d50 0x7fb2e4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd1fa640 1 -- 172.21.15.106:0/440098147 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e4003530 con 0x7fb2e40039c0 2024-04-25T11:20:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.615+0000 7fb2fd9fb640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2e4004d50 0x7fb2e4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:16.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2fd1fa640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e40039c0 0x7fb2e4003e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009ae0 tx=0x7fb2f401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838a325d3dba785a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f40b2070 con 0x7fb2e40039c0 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f40a5610 con 0x7fb2e40039c0 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2effff640 1 -- 172.21.15.106:0/440098147 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f40096b0 con 0x7fb2e40039c0 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2ff45e640 1 -- 172.21.15.106:0/440098147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e40039c0 msgr2=0x7fb2e4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2ff45e640 1 --2- 172.21.15.106:0/440098147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e40039c0 0x7fb2e4003e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009ae0 tx=0x7fb2f401ca20 comp rx=0 tx=0).stop 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2fd9fb640 1 -- 172.21.15.106:0/440098147 reap_dead start 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2ff45e640 1 -- 172.21.15.106:0/440098147 shutdown_connections 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2ff45e640 1 -- 172.21.15.106:0/440098147 >> 172.21.15.106:0/440098147 conn(0x7fb2f806c9f0 msgr2=0x7fb2f806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2ff45e640 1 -- 172.21.15.106:0/440098147 shutdown_connections 2024-04-25T11:20:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2ff45e640 1 -- 172.21.15.106:0/440098147 wait complete. 2024-04-25T11:20:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2ff45e640 1 Processor -- start 2024-04-25T11:20:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.619+0000 7fb2ff45e640 1 -- start start 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2e40039c0 0x7fb2f813fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f8163d10 0x7fb2f8161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f8161aa0 0x7fb2f8161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f8168b00 con 0x7fb2f8163d10 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f8168980 con 0x7fb2f8161aa0 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f8168c80 con 0x7fb2e40039c0 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f8163d10 0x7fb2f8161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f8163d10 0x7fb2f8161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59208/0 (socket says 172.21.15.106:59208) 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 -- 172.21.15.106:0/1776596620 learned_addr learned my addr 172.21.15.106:0/1776596620 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fd1fa640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2e40039c0 0x7fb2f813fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fd9fb640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f8161aa0 0x7fb2f8161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 -- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2e40039c0 msgr2=0x7fb2f813fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2e40039c0 0x7fb2f813fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 -- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f8161aa0 msgr2=0x7fb2f8161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f8161aa0 0x7fb2f8161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 -- 172.21.15.106:0/1776596620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f8142740 con 0x7fb2f8163d10 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fd9fb640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f8161aa0 0x7fb2f8161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2fc9f9640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f8163d10 0x7fb2f8161560 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800e7a0 tx=0x7fb2e800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2edffb640 1 -- 172.21.15.106:0/1776596620 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8017070 con 0x7fb2f8163d10 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2edffb640 1 -- 172.21.15.106:0/1776596620 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e800c940 con 0x7fb2f8163d10 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f815d2e0 con 0x7fb2f8163d10 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2edffb640 1 -- 172.21.15.106:0/1776596620 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800cd10 con 0x7fb2f8163d10 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f8140900 con 0x7fb2f8163d10 2024-04-25T11:20:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f8050750 con 0x7fb2f8163d10 2024-04-25T11:20:16.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2edffb640 1 -- 172.21.15.106:0/1776596620 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb2e801a070 con 0x7fb2f8163d10 2024-04-25T11:20:16.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.623+0000 7fb2edffb640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2dc06f430 0x7fb2dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.627+0000 7fb2edffb640 1 -- 172.21.15.106:0/1776596620 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb2e808f220 con 0x7fb2f8163d10 2024-04-25T11:20:16.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.627+0000 7fb2fd1fa640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2dc06f430 0x7fb2dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.627+0000 7fb2fd1fa640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2dc06f430 0x7fb2dc071940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40af000 tx=0x7fb2f40046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.631+0000 7fb2edffb640 1 -- 172.21.15.106:0/1776596620 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2e805c120 con 0x7fb2f8163d10 2024-04-25T11:20:16.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.779+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb2f80b4430 con 0x7fb2f8163d10 2024-04-25T11:20:16.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.779+0000 7fb2edffb640 1 -- 172.21.15.106:0/1776596620 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fb2e805ba20 con 0x7fb2f8163d10 2024-04-25T11:20:16.779 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:16.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.779+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2dc06f430 msgr2=0x7fb2dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.779+0000 7fb2ff45e640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2dc06f430 0x7fb2dc071940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40af000 tx=0x7fb2f40046d0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.779+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f8163d10 msgr2=0x7fb2f8161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.779+0000 7fb2ff45e640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f8163d10 0x7fb2f8161560 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800e7a0 tx=0x7fb2e800ec60 comp rx=0 tx=0).stop 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.783+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 shutdown_connections 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.783+0000 7fb2ff45e640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f8161aa0 0x7fb2f8161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.783+0000 7fb2ff45e640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb2dc06f430 0x7fb2dc071940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.783+0000 7fb2ff45e640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2f8163d10 0x7fb2f8161560 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.783+0000 7fb2ff45e640 1 --2- 172.21.15.106:0/1776596620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2e40039c0 0x7fb2f813fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.783+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 >> 172.21.15.106:0/1776596620 conn(0x7fb2f806c9f0 msgr2=0x7fb2f8145c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.783+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 shutdown_connections 2024-04-25T11:20:16.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.783+0000 7fb2ff45e640 1 -- 172.21.15.106:0/1776596620 wait complete. 2024-04-25T11:20:16.801 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:16.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa68d5fc640 1 Processor -- start 2024-04-25T11:20:16.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa68d5fc640 1 -- start start 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa68d5fc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa68806c820 0x7fa68806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa68d5fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa68806f7e0 0x7fa68806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa68d5fc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68806d170 con 0x7fa688073460 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa68d5fc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa68815f1a0 con 0x7fa68806f7e0 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa68d5fc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa688072fd0 con 0x7fa68806c820 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6877fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa68806f7e0 0x7fa68806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6877fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa68806f7e0 0x7fa68806fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59214/0 (socket says 172.21.15.106:59214) 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6877fe640 1 -- 172.21.15.106:0/2751392172 learned_addr learned my addr 172.21.15.106:0/2751392172 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6867fc640 1 --2- 172.21.15.106:0/2751392172 >> v2:172.21.15.134:3300/0 conn(0x7fa68806c820 0x7fa68806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6877fe640 1 -- 172.21.15.106:0/2751392172 >> v2:172.21.15.134:3300/0 conn(0x7fa68806c820 msgr2=0x7fa68806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6877fe640 1 --2- 172.21.15.106:0/2751392172 >> v2:172.21.15.134:3300/0 conn(0x7fa68806c820 0x7fa68806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6877fe640 1 -- 172.21.15.106:0/2751392172 >> v1:172.21.15.134:6789/0 conn(0x7fa688073460 legacy=0x7fa68815e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6877fe640 1 -- 172.21.15.106:0/2751392172 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa688144350 con 0x7fa68806f7e0 2024-04-25T11:20:16.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.935+0000 7fa6867fc640 1 --2- 172.21.15.106:0/2751392172 >> v2:172.21.15.134:3300/0 conn(0x7fa68806c820 0x7fa68806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa6877fe640 1 --2- 172.21.15.106:0/2751392172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa68806f7e0 0x7fa68806fbf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa6880b4510 tx=0x7fa68001c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341e1ac6786809a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa685ffb640 1 -- 172.21.15.106:0/2751392172 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800a5660 con 0x7fa68806f7e0 2024-04-25T11:20:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa685ffb640 1 -- 172.21.15.106:0/2751392172 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6800a5800 con 0x7fa68806f7e0 2024-04-25T11:20:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa685ffb640 1 -- 172.21.15.106:0/2751392172 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800a5bd0 con 0x7fa68806f7e0 2024-04-25T11:20:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa68d5fc640 1 -- 172.21.15.106:0/2751392172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa68806f7e0 msgr2=0x7fa68806fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/2751392172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa68806f7e0 0x7fa68806fbf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa6880b4510 tx=0x7fa68001c7a0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa68d5fc640 1 -- 172.21.15.106:0/2751392172 shutdown_connections 2024-04-25T11:20:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/2751392172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa68806f7e0 0x7fa68806fbf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/2751392172 >> v2:172.21.15.134:3300/0 conn(0x7fa68806c820 0x7fa68806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa68d5fc640 1 -- 172.21.15.106:0/2751392172 >> 172.21.15.106:0/2751392172 conn(0x7fa6880b3560 msgr2=0x7fa6880b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa68d5fc640 1 -- 172.21.15.106:0/2751392172 shutdown_connections 2024-04-25T11:20:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.939+0000 7fa68d5fc640 1 -- 172.21.15.106:0/2751392172 wait complete. 2024-04-25T11:20:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 Processor -- start 2024-04-25T11:20:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 -- start start 2024-04-25T11:20:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa68806c820 0x7fa688141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa68806f7e0 0x7fa68815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa688073460 0x7fa68815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa688073870 con 0x7fa688073460 2024-04-25T11:20:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa68815e8a0 con 0x7fa68806f7e0 2024-04-25T11:20:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa6880739f0 con 0x7fa68806c820 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa688073460 0x7fa68815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa688073460 0x7fa68815f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59226/0 (socket says 172.21.15.106:59226) 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 -- 172.21.15.106:0/734732149 learned_addr learned my addr 172.21.15.106:0/734732149 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 -- 172.21.15.106:0/734732149 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa68806c820 msgr2=0x7fa688141ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa68806c820 0x7fa688141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 -- 172.21.15.106:0/734732149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa68806f7e0 msgr2=0x7fa68815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6867fc640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa68806f7e0 0x7fa68815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa68806f7e0 0x7fa68815d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 -- 172.21.15.106:0/734732149 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa688143380 con 0x7fa688073460 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa6877fe640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa688073460 0x7fa68815f330 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa6800af040 tx=0x7fa680019540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa66f7fe640 1 -- 172.21.15.106:0/734732149 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800b8670 con 0x7fa688073460 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa66f7fe640 1 -- 172.21.15.106:0/734732149 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6800be070 con 0x7fa688073460 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa688143660 con 0x7fa688073460 2024-04-25T11:20:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa66f7fe640 1 -- 172.21.15.106:0/734732149 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800b8a50 con 0x7fa688073460 2024-04-25T11:20:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6881423e0 con 0x7fa688073460 2024-04-25T11:20:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.943+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa688133980 con 0x7fa688073460 2024-04-25T11:20:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.947+0000 7fa66f7fe640 1 -- 172.21.15.106:0/734732149 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa6800b8bf0 con 0x7fa688073460 2024-04-25T11:20:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.947+0000 7fa66f7fe640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa67406f400 0x7fa674071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.947+0000 7fa66f7fe640 1 -- 172.21.15.106:0/734732149 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa68012a4c0 con 0x7fa688073460 2024-04-25T11:20:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.947+0000 7fa686ffd640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa67406f400 0x7fa674071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.947+0000 7fa686ffd640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa67406f400 0x7fa674071910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa688070ae0 tx=0x7fa678006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:16.951+0000 7fa66f7fe640 1 -- 172.21.15.106:0/734732149 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa68806d170 con 0x7fa688073460 2024-04-25T11:20:17.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa688160120 con 0x7fa688073460 2024-04-25T11:20:17.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa66f7fe640 1 -- 172.21.15.106:0/734732149 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fa6800f7330 con 0x7fa688073460 2024-04-25T11:20:17.122 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa67406f400 msgr2=0x7fa674071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa67406f400 0x7fa674071910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa688070ae0 tx=0x7fa678006bd0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa688073460 msgr2=0x7fa68815f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa688073460 0x7fa68815f330 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa6800af040 tx=0x7fa680019540 comp rx=0 tx=0).stop 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 shutdown_connections 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa67406f400 0x7fa674071910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa688073460 0x7fa68815f330 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa68806f7e0 0x7fa68815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 --2- 172.21.15.106:0/734732149 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa68806c820 0x7fa688141ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 >> 172.21.15.106:0/734732149 conn(0x7fa6880b3560 msgr2=0x7fa68813fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 shutdown_connections 2024-04-25T11:20:17.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.123+0000 7fa68d5fc640 1 -- 172.21.15.106:0/734732149 wait complete. 2024-04-25T11:20:17.143 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi134.front.sepia.ceph.com... 2024-04-25T11:20:17.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:17.144 DEBUG:teuthology.orchestra.run.smithi134:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T11:20:17.159 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:17.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:20:17.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:17.159 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T11:20:17.209 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:20:17.209 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:17.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.359+0000 7f05d7169640 1 Processor -- start 2024-04-25T11:20:17.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.359+0000 7f05d7169640 1 -- start start 2024-04-25T11:20:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d7169640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f05d015e0a0 0x7f05d0073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d7169640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 0x7f05d00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d7169640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d00cf1a0 con 0x7f05d006f5f0 2024-04-25T11:20:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d7169640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f05d00d1cf0 con 0x7f05d015e0a0 2024-04-25T11:20:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d7169640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05d00ce050 con 0x7f05d00ce850 2024-04-25T11:20:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d5706640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f05d006f5f0 0x7f05d006fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49048/0 (socket says 172.21.15.106:49048) 2024-04-25T11:20:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d5706640 1 -- 172.21.15.106:0/786417818 learned_addr learned my addr 172.21.15.106:0/786417818 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738873728 0 0) 0x7f05c80014a0 con 0x7f05d006f5f0 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b8003530 con 0x7f05d006f5f0 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d4f05640 1 --2- 172.21.15.106:0/786417818 >> v2:172.21.15.134:3300/0 conn(0x7f05d015e0a0 0x7f05d0073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cffff640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 0x7f05d00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2938179439 0 0) 0x7f05b8003530 con 0x7f05d006f5f0 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 msgr2=0x7f05d00cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 0x7f05d00cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 >> v2:172.21.15.134:3300/0 conn(0x7f05d015e0a0 msgr2=0x7f05d0073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/786417818 >> v2:172.21.15.134:3300/0 conn(0x7f05d015e0a0 0x7f05d0073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d015e6b0 con 0x7f05d006f5f0 2024-04-25T11:20:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f05c8001bd0 con 0x7f05d006f5f0 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cffff640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 0x7f05d00cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 >> v1:172.21.15.106:6789/0 conn(0x7f05d006f5f0 legacy=0x7f05d006fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b80039c0 0x7f05b8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 0x7f05b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b8004d50 0x7f05b8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00ce050 con 0x7f05b80039c0 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05b800ba80 con 0x7f05b8004d50 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05b800bc80 con 0x7f05d00ce850 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d4f05640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b80039c0 0x7f05b8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05cffff640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 0x7f05b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d5706640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b8004d50 0x7f05b8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d4f05640 1 -- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 msgr2=0x7f05b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d4f05640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00ce850 0x7f05b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d4f05640 1 -- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b8004d50 msgr2=0x7f05b8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d4f05640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b8004d50 0x7f05b8005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.363+0000 7f05d4f05640 1 -- 172.21.15.106:0/786417818 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b8003530 con 0x7f05b80039c0 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05d4f05640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b80039c0 0x7f05b8003e30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f05c000ba00 tx=0x7f05c001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6803c6700216db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c000b600 con 0x7f05b80039c0 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c00b1070 con 0x7f05b80039c0 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05cf7fe640 1 -- 172.21.15.106:0/786417818 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c00ac860 con 0x7f05b80039c0 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05d7169640 1 -- 172.21.15.106:0/786417818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b80039c0 msgr2=0x7f05b8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05d7169640 1 --2- 172.21.15.106:0/786417818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b80039c0 0x7f05b8003e30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f05c000ba00 tx=0x7f05c001c6a0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05d5706640 1 -- 172.21.15.106:0/786417818 reap_dead start 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05d7169640 1 -- 172.21.15.106:0/786417818 shutdown_connections 2024-04-25T11:20:17.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05d7169640 1 -- 172.21.15.106:0/786417818 >> 172.21.15.106:0/786417818 conn(0x7f05d006c7c0 msgr2=0x7f05d006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05d7169640 1 -- 172.21.15.106:0/786417818 shutdown_connections 2024-04-25T11:20:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.367+0000 7f05d7169640 1 -- 172.21.15.106:0/786417818 wait complete. 2024-04-25T11:20:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 Processor -- start 2024-04-25T11:20:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 -- start start 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b80039c0 0x7f05d015f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d015d5a0 0x7f05d00b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d00b1b40 0x7f05d01fa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00d0860 con 0x7f05d015d5a0 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00d06e0 con 0x7f05d00b1b40 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05d015f270 con 0x7f05b80039c0 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d015d5a0 0x7f05d00b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d015d5a0 0x7f05d00b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59234/0 (socket says 172.21.15.106:59234) 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 -- 172.21.15.106:0/3136545360 learned_addr learned my addr 172.21.15.106:0/3136545360 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d5706640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d00b1b40 0x7f05d01fa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d4f05640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b80039c0 0x7f05d015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 -- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b80039c0 msgr2=0x7f05d015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b80039c0 0x7f05d015f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 -- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d00b1b40 msgr2=0x7f05d01fa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d00b1b40 0x7f05d01fa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 -- 172.21.15.106:0/3136545360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d00b1f50 con 0x7f05d015d5a0 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d4f05640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b80039c0 0x7f05d015f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cffff640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d015d5a0 0x7f05d00b1600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f05c40108c0 tx=0x7f05c4010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cd7fa640 1 -- 172.21.15.106:0/3136545360 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c401a020 con 0x7f05d015d5a0 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cd7fa640 1 -- 172.21.15.106:0/3136545360 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c400cb40 con 0x7f05d015d5a0 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d015ffb0 con 0x7f05d015d5a0 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05cd7fa640 1 -- 172.21.15.106:0/3136545360 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c400e050 con 0x7f05d015d5a0 2024-04-25T11:20:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.371+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d01f8530 con 0x7f05d015d5a0 2024-04-25T11:20:17.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.375+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d01604d0 con 0x7f05d015d5a0 2024-04-25T11:20:17.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.375+0000 7f05cd7fa640 1 -- 172.21.15.106:0/3136545360 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f05c401d020 con 0x7f05d015d5a0 2024-04-25T11:20:17.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.375+0000 7f05cd7fa640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05bc06f400 0x7f05bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.375+0000 7f05d4f05640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05bc06f400 0x7f05bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.375+0000 7f05cd7fa640 1 -- 172.21.15.106:0/3136545360 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f05c408fab0 con 0x7f05d015d5a0 2024-04-25T11:20:17.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.375+0000 7f05d4f05640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05bc06f400 0x7f05bc071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f05c00b0000 tx=0x7f05c00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.379+0000 7f05cd7fa640 1 -- 172.21.15.106:0/3136545360 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f05d01604d0 con 0x7f05d015d5a0 2024-04-25T11:20:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05d015fd40 con 0x7f05d015d5a0 2024-04-25T11:20:17.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05cd7fa640 1 -- 172.21.15.106:0/3136545360 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f05c4024020 con 0x7f05d015d5a0 2024-04-25T11:20:17.563 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:17.563 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:26.806158+0000","modified":"2024-04-25T11:19:29.823480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:20:17.563 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05bc06f400 msgr2=0x7f05bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05bc06f400 0x7f05bc071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f05c00b0000 tx=0x7f05c00046d0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d015d5a0 msgr2=0x7f05d00b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d015d5a0 0x7f05d00b1600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f05c40108c0 tx=0x7f05c4010d80 comp rx=0 tx=0).stop 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 shutdown_connections 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d00b1b40 0x7f05d01fa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f05bc06f400 0x7f05bc071910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d015d5a0 0x7f05d00b1600 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 --2- 172.21.15.106:0/3136545360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b80039c0 0x7f05d015f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 >> 172.21.15.106:0/3136545360 conn(0x7f05d006c7c0 msgr2=0x7f05d0167ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 shutdown_connections 2024-04-25T11:20:17.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.563+0000 7f05d7169640 1 -- 172.21.15.106:0/3136545360 wait complete. 2024-04-25T11:20:17.583 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:17.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbab4dfd640 1 Processor -- start 2024-04-25T11:20:17.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbab4dfd640 1 -- start start 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbab4dfd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbab00ba970 0x7fbab00d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbab4dfd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 0x7fbab00d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbab4dfd640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab00d2ab0 con 0x7fbab00b5120 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbab4dfd640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbab00d1100 con 0x7fbab00ba970 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbab4dfd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbab00d0530 con 0x7fbab00d2160 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaae59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbab00ba970 0x7fbab00d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaae59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbab00ba970 0x7fbab00d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59242/0 (socket says 172.21.15.106:59242) 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaae59c640 1 -- 172.21.15.106:0/3640893844 learned_addr learned my addr 172.21.15.106:0/3640893844 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaadd9b640 1 --2- 172.21.15.106:0/3640893844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 0x7fbab00d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaae59c640 1 -- 172.21.15.106:0/3640893844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 msgr2=0x7fbab00d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaae59c640 1 --2- 172.21.15.106:0/3640893844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 0x7fbab00d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaae59c640 1 -- 172.21.15.106:0/3640893844 >> v1:172.21.15.134:6789/0 conn(0x7fbab00b5120 legacy=0x7fbab00b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaae59c640 1 -- 172.21.15.106:0/3640893844 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab00b5c40 con 0x7fbab00ba970 2024-04-25T11:20:17.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.715+0000 7fbaadd9b640 1 --2- 172.21.15.106:0/3640893844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 0x7fbab00d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:17.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbaae59c640 1 --2- 172.21.15.106:0/3640893844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00ba970 0x7fbab00d1590 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0006fd0 tx=0x7fbaa001c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92efe8aebd6d31da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbaad59a640 1 -- 172.21.15.106:0/3640893844 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00b0070 con 0x7fbab00ba970 2024-04-25T11:20:17.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbaad59a640 1 -- 172.21.15.106:0/3640893844 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa00ab600 con 0x7fbab00ba970 2024-04-25T11:20:17.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbaad59a640 1 -- 172.21.15.106:0/3640893844 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00ab9d0 con 0x7fbab00ba970 2024-04-25T11:20:17.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3640893844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00ba970 msgr2=0x7fbab00d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3640893844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00ba970 0x7fbab00d1590 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0006fd0 tx=0x7fbaa001c320 comp rx=0 tx=0).stop 2024-04-25T11:20:17.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3640893844 shutdown_connections 2024-04-25T11:20:17.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3640893844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 0x7fbab00d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3640893844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00ba970 0x7fbab00d1590 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3640893844 >> 172.21.15.106:0/3640893844 conn(0x7fbab00d57e0 msgr2=0x7fbab00d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3640893844 shutdown_connections 2024-04-25T11:20:17.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3640893844 wait complete. 2024-04-25T11:20:17.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.719+0000 7fbab4dfd640 1 Processor -- start 2024-04-25T11:20:17.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 -- start start 2024-04-25T11:20:17.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00b5120 0x7fbab015cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbab00ba970 0x7fbab00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 0x7fbab00ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbab00b4da0 con 0x7fbab00b5120 2024-04-25T11:20:17.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbab00d1ad0 con 0x7fbab00ba970 2024-04-25T11:20:17.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbab00b4f20 con 0x7fbab00d2160 2024-04-25T11:20:17.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00b5120 0x7fbab015cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00b5120 0x7fbab015cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59258/0 (socket says 172.21.15.106:59258) 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 -- 172.21.15.106:0/3735447341 learned_addr learned my addr 172.21.15.106:0/3735447341 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaadd9b640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbab00ba970 0x7fbab00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 -- 172.21.15.106:0/3735447341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 msgr2=0x7fbab00ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 0x7fbab00ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 -- 172.21.15.106:0/3735447341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbab00ba970 msgr2=0x7fbab00c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbab00ba970 0x7fbab00c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 -- 172.21.15.106:0/3735447341 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab01f9a40 con 0x7fbab00b5120 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbaae59c640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00b5120 0x7fbab015cce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbaa001a9d0 tx=0x7fbaa00190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fba9effd640 1 -- 172.21.15.106:0/3735447341 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00bb070 con 0x7fbab00b5120 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fba9effd640 1 -- 172.21.15.106:0/3735447341 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa00ab9b0 con 0x7fbab00b5120 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab00caa10 con 0x7fbab00b5120 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fba9effd640 1 -- 172.21.15.106:0/3735447341 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00b0040 con 0x7fbab00b5120 2024-04-25T11:20:17.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab00cafd0 con 0x7fbab00b5120 2024-04-25T11:20:17.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab00d2ed0 con 0x7fbab00b5120 2024-04-25T11:20:17.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.723+0000 7fba9effd640 1 -- 172.21.15.106:0/3735447341 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbaa00be070 con 0x7fbab00b5120 2024-04-25T11:20:17.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.727+0000 7fba9effd640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fba9006f430 0x7fba90071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.727+0000 7fba9effd640 1 -- 172.21.15.106:0/3735447341 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbaa0128be0 con 0x7fbab00b5120 2024-04-25T11:20:17.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.727+0000 7fbaadd9b640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fba9006f430 0x7fba90071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.727+0000 7fbaadd9b640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fba9006f430 0x7fba90071940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbab01fcf10 tx=0x7fbaa4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.731+0000 7fba9effd640 1 -- 172.21.15.106:0/3735447341 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbab00d2ab0 con 0x7fbab00b5120 2024-04-25T11:20:17.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.875+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbab00b2720 con 0x7fbab00b5120 2024-04-25T11:20:17.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fba9effd640 1 -- 172.21.15.106:0/3735447341 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16834 (secure 0 0 0) 0x7fbaa00f5bc0 con 0x7fbab00b5120 2024-04-25T11:20:17.879 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:17.879 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:07.237284+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:17.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fba9006f430 msgr2=0x7fba90071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fba9006f430 0x7fba90071940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbab01fcf10 tx=0x7fbaa4008040 comp rx=0 tx=0).stop 2024-04-25T11:20:17.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00b5120 msgr2=0x7fbab015cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00b5120 0x7fbab015cce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbaa001a9d0 tx=0x7fbaa00190c0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 shutdown_connections 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbab00d2160 0x7fbab00ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbab00ba970 0x7fbab00c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fba9006f430 0x7fba90071940 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 --2- 172.21.15.106:0/3735447341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab00b5120 0x7fbab015cce0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 >> 172.21.15.106:0/3735447341 conn(0x7fbab00d57e0 msgr2=0x7fbab00b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 shutdown_connections 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:17.879+0000 7fbab4dfd640 1 -- 172.21.15.106:0/3735447341 wait complete. 2024-04-25T11:20:17.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:24.592862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:25.761448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:17.900 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.047+0000 7fb5cea77640 1 Processor -- start 2024-04-25T11:20:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.047+0000 7fb5cea77640 1 -- start start 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cea77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c80cd930 0x7fb5c80d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cea77640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c80cdd40 con 0x7fb5c8143fa0 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cea77640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c80d2090 con 0x7fb5c80d1c80 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cea77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c8073300 con 0x7fb5c80cd930 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cc813640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb5c8143fa0 0x7fb5c8072a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49060/0 (socket says 172.21.15.106:49060) 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cc813640 1 -- 172.21.15.106:0/2644461132 learned_addr learned my addr 172.21.15.106:0/2644461132 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c7fff640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c80cd930 0x7fb5c80d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987354492 0 0) 0x7fb5b8001120 con 0x7fb5c8143fa0 2024-04-25T11:20:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5b0003530 con 0x7fb5c8143fa0 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 91712230 0 0) 0x7fb5b0003530 con 0x7fb5c8143fa0 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c80cd930 msgr2=0x7fb5c80d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c80cd930 0x7fb5c80d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 >> v1:172.21.15.134:6789/0 conn(0x7fb5c80d1c80 legacy=0x7fb5c80b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c80735e0 con 0x7fb5c8143fa0 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fb5b8001850 con 0x7fb5c8143fa0 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 >> v1:172.21.15.106:6789/0 conn(0x7fb5c8143fa0 legacy=0x7fb5c8072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5b00039f0 0x7fb5b0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c80cd930 0x7fb5b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b0004d50 0x7fb5b0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c8073300 con 0x7fb5b00039f0 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b000bb00 con 0x7fb5b0004d50 2024-04-25T11:20:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b000bd00 con 0x7fb5c80cd930 2024-04-25T11:20:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cc813640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5b00039f0 0x7fb5b0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cd014640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b0004d50 0x7fb5b0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cc813640 1 -- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c80cd930 msgr2=0x7fb5b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cc813640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c80cd930 0x7fb5b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cc813640 1 -- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b0004d50 msgr2=0x7fb5b0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cc813640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b0004d50 0x7fb5b0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cc813640 1 -- 172.21.15.106:0/2644461132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b0003530 con 0x7fb5b00039f0 2024-04-25T11:20:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.051+0000 7fb5cd014640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b0004d50 0x7fb5b0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cc813640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5b00039f0 0x7fb5b0003e60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c4c0 tx=0x7fb5b801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b2c406f5fc95e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800c0e0 con 0x7fb5b00039f0 2024-04-25T11:20:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b80a9070 con 0x7fb5b00039f0 2024-04-25T11:20:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5c77fe640 1 -- 172.21.15.106:0/2644461132 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8005d40 con 0x7fb5b00039f0 2024-04-25T11:20:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cea77640 1 -- 172.21.15.106:0/2644461132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5b00039f0 msgr2=0x7fb5b0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cea77640 1 --2- 172.21.15.106:0/2644461132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5b00039f0 0x7fb5b0003e60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c4c0 tx=0x7fb5b801cbc0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cd014640 1 -- 172.21.15.106:0/2644461132 reap_dead start 2024-04-25T11:20:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cea77640 1 -- 172.21.15.106:0/2644461132 shutdown_connections 2024-04-25T11:20:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cea77640 1 -- 172.21.15.106:0/2644461132 >> 172.21.15.106:0/2644461132 conn(0x7fb5c806c9f0 msgr2=0x7fb5c806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cea77640 1 -- 172.21.15.106:0/2644461132 shutdown_connections 2024-04-25T11:20:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cea77640 1 -- 172.21.15.106:0/2644461132 wait complete. 2024-04-25T11:20:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cea77640 1 Processor -- start 2024-04-25T11:20:18.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.055+0000 7fb5cea77640 1 -- start start 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b00039f0 0x7fb5c8159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5c8146890 0x7fb5c8142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c8166970 0x7fb5c8166d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c8169b10 con 0x7fb5c8146890 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c8169990 con 0x7fb5b00039f0 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c8169c90 con 0x7fb5c8166970 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5c8146890 0x7fb5c8142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5c8146890 0x7fb5c8142d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59264/0 (socket says 172.21.15.106:59264) 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 -- 172.21.15.106:0/356249615 learned_addr learned my addr 172.21.15.106:0/356249615 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cd014640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c8166970 0x7fb5c8166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cc813640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b00039f0 0x7fb5c8159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 -- 172.21.15.106:0/356249615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c8166970 msgr2=0x7fb5c8166d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c8166970 0x7fb5c8166d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 -- 172.21.15.106:0/356249615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b00039f0 msgr2=0x7fb5c8159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b00039f0 0x7fb5c8159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 -- 172.21.15.106:0/356249615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c81672c0 con 0x7fb5c8146890 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c7fff640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5c8146890 0x7fb5c8142d50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e7d0 tx=0x7fb5bc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c57fa640 1 -- 172.21.15.106:0/356249615 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc017070 con 0x7fb5c8146890 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c57fa640 1 -- 172.21.15.106:0/356249615 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5bc00c940 con 0x7fb5c8146890 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c815a790 con 0x7fb5c8146890 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c57fa640 1 -- 172.21.15.106:0/356249615 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc00cd10 con 0x7fb5c8146890 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c815ad50 con 0x7fb5c8146890 2024-04-25T11:20:18.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c8050750 con 0x7fb5c8146890 2024-04-25T11:20:18.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c57fa640 1 -- 172.21.15.106:0/356249615 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5bc01a070 con 0x7fb5c8146890 2024-04-25T11:20:18.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c57fa640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb5b406f430 0x7fb5b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.059+0000 7fb5c57fa640 1 -- 172.21.15.106:0/356249615 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb5bc08f0e0 con 0x7fb5c8146890 2024-04-25T11:20:18.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.063+0000 7fb5cc813640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb5b406f430 0x7fb5b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.063+0000 7fb5cc813640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb5b406f430 0x7fb5b4071940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80ab000 tx=0x7fb5b80b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.067+0000 7fb5c57fa640 1 -- 172.21.15.106:0/356249615 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb5c81672c0 con 0x7fb5c8146890 2024-04-25T11:20:18.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5c8158880 con 0x7fb5c8146890 2024-04-25T11:20:18.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5c57fa640 1 -- 172.21.15.106:0/356249615 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fb5bc05bf50 con 0x7fb5c8146890 2024-04-25T11:20:18.227 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb5b406f430 msgr2=0x7fb5b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb5b406f430 0x7fb5b4071940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80ab000 tx=0x7fb5b80b1040 comp rx=0 tx=0).stop 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5c8146890 msgr2=0x7fb5c8142d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5c8146890 0x7fb5c8142d50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e7d0 tx=0x7fb5bc00ec90 comp rx=0 tx=0).stop 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 shutdown_connections 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5c8166970 0x7fb5c8166d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb5b406f430 0x7fb5b4071940 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5c8146890 0x7fb5c8142d50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.227+0000 7fb5cea77640 1 --2- 172.21.15.106:0/356249615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b00039f0 0x7fb5c8159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.231+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 >> 172.21.15.106:0/356249615 conn(0x7fb5c806c9f0 msgr2=0x7fb5c8158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.231+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 shutdown_connections 2024-04-25T11:20:18.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.231+0000 7fb5cea77640 1 -- 172.21.15.106:0/356249615 wait complete. 2024-04-25T11:20:18.249 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:18.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.399+0000 7f3cf5f58640 1 Processor -- start 2024-04-25T11:20:18.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cf5f58640 1 -- start start 2024-04-25T11:20:18.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cf5f58640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cf0143f10 0x7f3cf00d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cf5f58640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00d1740 0x7f3cf00d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cf5f58640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf00d2090 con 0x7f3cf00b5520 2024-04-25T11:20:18.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cf5f58640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf0144320 con 0x7f3cf0143f10 2024-04-25T11:20:18.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cf5f58640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf0161670 con 0x7f3cf00d1740 2024-04-25T11:20:18.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cef7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cf0143f10 0x7f3cf00d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cef7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cf0143f10 0x7f3cf00d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59278/0 (socket says 172.21.15.106:59278) 2024-04-25T11:20:18.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cef7fe640 1 -- 172.21.15.106:0/4232654427 learned_addr learned my addr 172.21.15.106:0/4232654427 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:18.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3ceeffd640 1 --2- 172.21.15.106:0/4232654427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00d1740 0x7f3cf00d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cef7fe640 1 -- 172.21.15.106:0/4232654427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00d1740 msgr2=0x7f3cf00d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cef7fe640 1 --2- 172.21.15.106:0/4232654427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00d1740 0x7f3cf00d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cef7fe640 1 -- 172.21.15.106:0/4232654427 >> v1:172.21.15.106:6789/0 conn(0x7f3cf00b5520 legacy=0x7f3cf00b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.403+0000 7f3cef7fe640 1 -- 172.21.15.106:0/4232654427 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf00d0fa0 con 0x7f3cf0143f10 2024-04-25T11:20:18.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cef7fe640 1 --2- 172.21.15.106:0/4232654427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf0143f10 0x7f3cf00d05c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009870 tx=0x7f3ce001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78d64260921c22a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cee7fc640 1 -- 172.21.15.106:0/4232654427 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00b0070 con 0x7f3cf0143f10 2024-04-25T11:20:18.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cee7fc640 1 -- 172.21.15.106:0/4232654427 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce00ab600 con 0x7f3cf0143f10 2024-04-25T11:20:18.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cee7fc640 1 -- 172.21.15.106:0/4232654427 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00ab9d0 con 0x7f3cf0143f10 2024-04-25T11:20:18.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cf5f58640 1 -- 172.21.15.106:0/4232654427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf0143f10 msgr2=0x7f3cf00d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/4232654427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf0143f10 0x7f3cf00d05c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009870 tx=0x7f3ce001c440 comp rx=0 tx=0).stop 2024-04-25T11:20:18.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cf5f58640 1 -- 172.21.15.106:0/4232654427 shutdown_connections 2024-04-25T11:20:18.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/4232654427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00d1740 0x7f3cf00d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/4232654427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf0143f10 0x7f3cf00d05c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cf5f58640 1 -- 172.21.15.106:0/4232654427 >> 172.21.15.106:0/4232654427 conn(0x7f3cf006c9f0 msgr2=0x7f3cf006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cf5f58640 1 -- 172.21.15.106:0/4232654427 shutdown_connections 2024-04-25T11:20:18.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.407+0000 7f3cf5f58640 1 -- 172.21.15.106:0/4232654427 wait complete. 2024-04-25T11:20:18.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 Processor -- start 2024-04-25T11:20:18.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 -- start start 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00b5520 0x7f3cf0140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf00d1740 0x7f3cf01405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cf0143f10 0x7f3cf013b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf00d0c80 con 0x7f3cf00d1740 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf00d0b00 con 0x7f3cf0143f10 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf01623a0 con 0x7f3cf00b5520 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf00d1740 0x7f3cf01405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf00d1740 0x7f3cf01405c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59286/0 (socket says 172.21.15.106:59286) 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 -- 172.21.15.106:0/2214372306 learned_addr learned my addr 172.21.15.106:0/2214372306 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:18.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cef7fe640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00b5520 0x7f3cf0140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceffff640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cf0143f10 0x7f3cf013b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 -- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00b5520 msgr2=0x7f3cf0140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00b5520 0x7f3cf0140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 -- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cf0143f10 msgr2=0x7f3cf013b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cf0143f10 0x7f3cf013b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 -- 172.21.15.106:0/2214372306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf0160320 con 0x7f3cf00d1740 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cef7fe640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00b5520 0x7f3cf0140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceffff640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cf0143f10 0x7f3cf013b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3ceeffd640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf00d1740 0x7f3cf01405c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400d650 tx=0x7f3ce400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cd7fff640 1 -- 172.21.15.106:0/2214372306 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce400b6c0 con 0x7f3cf00d1740 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cd7fff640 1 -- 172.21.15.106:0/2214372306 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce4019070 con 0x7f3cf00d1740 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf0160600 con 0x7f3cf00d1740 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cd7fff640 1 -- 172.21.15.106:0/2214372306 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce400ba80 con 0x7f3cf00d1740 2024-04-25T11:20:18.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.411+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf0145a10 con 0x7f3cf00d1740 2024-04-25T11:20:18.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.415+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf0075fe0 con 0x7f3cf00d1740 2024-04-25T11:20:18.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.415+0000 7f3cd7fff640 1 -- 172.21.15.106:0/2214372306 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3ce400bc20 con 0x7f3cf00d1740 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.415+0000 7f3cd7fff640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3cdc06f430 0x7f3cdc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.415+0000 7f3cd7fff640 1 -- 172.21.15.106:0/2214372306 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3ce408f5f0 con 0x7f3cf00d1740 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.415+0000 7f3cef7fe640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3cdc06f430 0x7f3cdc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.415+0000 7f3cef7fe640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3cdc06f430 0x7f3cdc071940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00ad000 tx=0x7f3ce0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.419+0000 7f3cd7fff640 1 -- 172.21.15.106:0/2214372306 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ce405c580 con 0x7f3cf00d1740 2024-04-25T11:20:18.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.567+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3cf0050750 con 0x7f3cf00d1740 2024-04-25T11:20:18.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cd7fff640 1 -- 172.21.15.106:0/2214372306 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f3ce405be80 con 0x7f3cf00d1740 2024-04-25T11:20:18.570 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3cdc06f430 msgr2=0x7f3cdc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3cdc06f430 0x7f3cdc071940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00ad000 tx=0x7f3ce0008040 comp rx=0 tx=0).stop 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf00d1740 msgr2=0x7f3cf01405c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf00d1740 0x7f3cf01405c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400d650 tx=0x7f3ce400db10 comp rx=0 tx=0).stop 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 shutdown_connections 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cf0143f10 0x7f3cf013b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3cdc06f430 0x7f3cdc071940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cf00d1740 0x7f3cf01405c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 --2- 172.21.15.106:0/2214372306 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cf00b5520 0x7f3cf0140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 >> 172.21.15.106:0/2214372306 conn(0x7f3cf006c9f0 msgr2=0x7f3cf013de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 shutdown_connections 2024-04-25T11:20:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.571+0000 7f3cf5f58640 1 -- 172.21.15.106:0/2214372306 wait complete. 2024-04-25T11:20:18.590 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:18.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.735+0000 7f3f080fa640 1 Processor -- start 2024-04-25T11:20:18.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.735+0000 7f3f080fa640 1 -- start start 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.735+0000 7f3f080fa640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3f000b6dd0 0x7f3f000b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.735+0000 7f3f080fa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f000d1520 con 0x7f3f0009ca40 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.735+0000 7f3f080fa640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f000cf370 con 0x7f3f00166230 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.735+0000 7f3f080fa640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3f000d1aa0 con 0x7f3f000b6dd0 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.735+0000 7f3f05e96640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3f0009ca40 0x7f3f000d1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49078/0 (socket says 172.21.15.106:49078) 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.735+0000 7f3f05e96640 1 -- 172.21.15.106:0/4109510062 learned_addr learned my addr 172.21.15.106:0/4109510062 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f05695640 1 --2- 172.21.15.106:0/4109510062 >> v2:172.21.15.134:3300/0 conn(0x7f3f000b6dd0 0x7f3f000b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2542347476 0 0) 0x7f3f000d1520 con 0x7f3f0009ca40 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ee8003530 con 0x7f3f0009ca40 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1484687640 0 0) 0x7f3ee8003530 con 0x7f3f0009ca40 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 >> v2:172.21.15.134:3300/0 conn(0x7f3f000b6dd0 msgr2=0x7f3f000b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 --2- 172.21.15.106:0/4109510062 >> v2:172.21.15.134:3300/0 conn(0x7f3f000b6dd0 0x7f3f000b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 >> v1:172.21.15.134:6789/0 conn(0x7f3f00166230 legacy=0x7f3f00168650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f000b8d80 con 0x7f3f0009ca40 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f3ef8001b80 con 0x7f3f0009ca40 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 >> v1:172.21.15.106:6789/0 conn(0x7f3f0009ca40 legacy=0x7f3f000d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee80039c0 0x7f3ee8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee8004810 0x7f3ee8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee8005260 0x7f3ee800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000d1aa0 con 0x7f3ee8005260 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee800c1f0 con 0x7f3ee80039c0 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee800c3f0 con 0x7f3ee8004810 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f06697640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee8005260 0x7f3ee800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f05695640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee8004810 0x7f3ee8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f05e96640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee80039c0 0x7f3ee8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f06697640 1 -- 172.21.15.106:0/4109510062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee8004810 msgr2=0x7f3ee8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f06697640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee8004810 0x7f3ee8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f06697640 1 -- 172.21.15.106:0/4109510062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee80039c0 msgr2=0x7f3ee8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f06697640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee80039c0 0x7f3ee8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f06697640 1 -- 172.21.15.106:0/4109510062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee8003530 con 0x7f3ee8005260 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f06697640 1 -- 172.21.15.106:0/4109510062 reap_dead start 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f06697640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee8005260 0x7f3ee800b9d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0001750 tx=0x7f3ef001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d577a6096e9046b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00013a0 con 0x7f3ee8005260 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef00052d0 con 0x7f3ee8005260 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.739+0000 7f3f04e94640 1 -- 172.21.15.106:0/4109510062 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0003820 con 0x7f3ee8005260 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- 172.21.15.106:0/4109510062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee8005260 msgr2=0x7f3ee800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee8005260 0x7f3ee800b9d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0001750 tx=0x7f3ef001c3e0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- 172.21.15.106:0/4109510062 shutdown_connections 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 --2- 172.21.15.106:0/4109510062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee8005260 0x7f3ee800b9d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- 172.21.15.106:0/4109510062 >> 172.21.15.106:0/4109510062 conn(0x7f3f000cfff0 msgr2=0x7f3f000d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- 172.21.15.106:0/4109510062 shutdown_connections 2024-04-25T11:20:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- 172.21.15.106:0/4109510062 wait complete. 2024-04-25T11:20:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 Processor -- start 2024-04-25T11:20:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- start start 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee8005260 0x7f3f000c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f00156c30 0x7f3f000b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f000b1350 0x7f3f000b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000cf570 con 0x7f3f000b1350 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b9050 con 0x7f3ee8005260 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.743+0000 7f3f080fa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000c6d20 con 0x7f3f00156c30 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f000b1350 0x7f3f000b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f000b1350 0x7f3f000b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59302/0 (socket says 172.21.15.106:59302) 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 -- 172.21.15.106:0/1841990559 learned_addr learned my addr 172.21.15.106:0/1841990559 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f05695640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f00156c30 0x7f3f000b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f05e96640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee8005260 0x7f3f000c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 -- 172.21.15.106:0/1841990559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f00156c30 msgr2=0x7f3f000b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f00156c30 0x7f3f000b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 -- 172.21.15.106:0/1841990559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee8005260 msgr2=0x7f3f000c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee8005260 0x7f3f000c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 -- 172.21.15.106:0/1841990559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f00165010 con 0x7f3f000b1350 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f06697640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f000b1350 0x7f3f000b1760 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0001750 tx=0x7f3ee8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3efe7fc640 1 -- 172.21.15.106:0/1841990559 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00013a0 con 0x7f3f000b1350 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3efe7fc640 1 -- 172.21.15.106:0/1841990559 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0005480 con 0x7f3f000b1350 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f000c7270 con 0x7f3f000b1350 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3efe7fc640 1 -- 172.21.15.106:0/1841990559 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00067e0 con 0x7f3f000b1350 2024-04-25T11:20:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f000c7830 con 0x7f3f000b1350 2024-04-25T11:20:18.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f00050750 con 0x7f3f000b1350 2024-04-25T11:20:18.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3efe7fc640 1 -- 172.21.15.106:0/1841990559 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3ef0006980 con 0x7f3f000b1350 2024-04-25T11:20:18.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3efe7fc640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3eec06f400 0x7f3eec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f05e96640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3eec06f400 0x7f3eec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3efe7fc640 1 -- 172.21.15.106:0/1841990559 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3ef01288d0 con 0x7f3f000b1350 2024-04-25T11:20:18.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.747+0000 7f3f05e96640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3eec06f400 0x7f3eec071910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8001280 tx=0x7f3ef8002180 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.751+0000 7f3efe7fc640 1 -- 172.21.15.106:0/1841990559 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3f00165010 con 0x7f3f000b1350 2024-04-25T11:20:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.911+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f00076010 con 0x7f3f000b1350 2024-04-25T11:20:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.911+0000 7f3efe7fc640 1 -- 172.21.15.106:0/1841990559 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f3ef00160f0 con 0x7f3f000b1350 2024-04-25T11:20:18.912 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3eec06f400 msgr2=0x7f3eec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3eec06f400 0x7f3eec071910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8001280 tx=0x7f3ef8002180 comp rx=0 tx=0).stop 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f000b1350 msgr2=0x7f3f000b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f000b1350 0x7f3f000b1760 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0001750 tx=0x7f3ee8005840 comp rx=0 tx=0).stop 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 shutdown_connections 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3eec06f400 0x7f3eec071910 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f000b1350 0x7f3f000b1760 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f00156c30 0x7f3f000b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 --2- 172.21.15.106:0/1841990559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee8005260 0x7f3f000c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 >> 172.21.15.106:0/1841990559 conn(0x7f3f000cfff0 msgr2=0x7f3f00071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 shutdown_connections 2024-04-25T11:20:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:18.915+0000 7f3f080fa640 1 -- 172.21.15.106:0/1841990559 wait complete. 2024-04-25T11:20:18.933 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:19.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9d151d640 1 Processor -- start 2024-04-25T11:20:19.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9d151d640 1 -- start start 2024-04-25T11:20:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9d151d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15e0a0 0x7fe9cc073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9d151d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9cc0d0ce0 0x7fe9cc0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9d151d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9cc0ce050 con 0x7fe9cc0ce850 2024-04-25T11:20:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9d151d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe9cc15e6b0 con 0x7fe9cc0d0ce0 2024-04-25T11:20:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9d151d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9cc15e990 con 0x7fe9cc15e0a0 2024-04-25T11:20:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9cb7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9cc0d0ce0 0x7fe9cc0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9caffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15e0a0 0x7fe9cc073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9ca7fc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe9cc0ce850 0x7fe9cc0cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:54980/0 (socket says 172.21.15.106:54980) 2024-04-25T11:20:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9ca7fc640 1 -- 172.21.15.106:0/2717822093 learned_addr learned my addr 172.21.15.106:0/2717822093 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9caffd640 1 -- 172.21.15.106:0/2717822093 >> v2:172.21.15.134:3300/0 conn(0x7fe9cc0d0ce0 msgr2=0x7fe9cc0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9caffd640 1 --2- 172.21.15.106:0/2717822093 >> v2:172.21.15.134:3300/0 conn(0x7fe9cc0d0ce0 0x7fe9cc0d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9caffd640 1 -- 172.21.15.106:0/2717822093 >> v1:172.21.15.134:6789/0 conn(0x7fe9cc0ce850 legacy=0x7fe9cc0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.083+0000 7fe9caffd640 1 -- 172.21.15.106:0/2717822093 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc15ec70 con 0x7fe9cc15e0a0 2024-04-25T11:20:19.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9caffd640 1 --2- 172.21.15.106:0/2717822093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15e0a0 0x7fe9cc073460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0097c0 tx=0x7fe9bc01c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cedd9bc9b2ca896d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9c9ffb640 1 -- 172.21.15.106:0/2717822093 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0b0070 con 0x7fe9cc15e0a0 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9c9ffb640 1 -- 172.21.15.106:0/2717822093 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9bc0ab600 con 0x7fe9cc15e0a0 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9c9ffb640 1 -- 172.21.15.106:0/2717822093 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0ab9d0 con 0x7fe9cc15e0a0 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9d151d640 1 -- 172.21.15.106:0/2717822093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15e0a0 msgr2=0x7fe9cc073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9d151d640 1 --2- 172.21.15.106:0/2717822093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15e0a0 0x7fe9cc073460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0097c0 tx=0x7fe9bc01c460 comp rx=0 tx=0).stop 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9d151d640 1 -- 172.21.15.106:0/2717822093 shutdown_connections 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9d151d640 1 --2- 172.21.15.106:0/2717822093 >> v2:172.21.15.134:3300/0 conn(0x7fe9cc0d0ce0 0x7fe9cc0d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9d151d640 1 --2- 172.21.15.106:0/2717822093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15e0a0 0x7fe9cc073460 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0097c0 tx=0x7fe9bc01c460 comp rx=0 tx=0).stop 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9d151d640 1 -- 172.21.15.106:0/2717822093 >> 172.21.15.106:0/2717822093 conn(0x7fe9cc06c7c0 msgr2=0x7fe9cc06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9d151d640 1 -- 172.21.15.106:0/2717822093 shutdown_connections 2024-04-25T11:20:19.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.087+0000 7fe9d151d640 1 -- 172.21.15.106:0/2717822093 wait complete. 2024-04-25T11:20:19.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 Processor -- start 2024-04-25T11:20:19.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 -- start start 2024-04-25T11:20:19.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc0ce850 0x7fe9cc15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc0d0ce0 0x7fe9cc141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15ba90 0x7fe9cc15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc16a870 con 0x7fe9cc0ce850 2024-04-25T11:20:19.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc16a6f0 con 0x7fe9cc0d0ce0 2024-04-25T11:20:19.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9cc16a9f0 con 0x7fe9cc15ba90 2024-04-25T11:20:19.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc0ce850 0x7fe9cc15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc0ce850 0x7fe9cc15b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59308/0 (socket says 172.21.15.106:59308) 2024-04-25T11:20:19.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 -- 172.21.15.106:0/4241546723 learned_addr learned my addr 172.21.15.106:0/4241546723 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:19.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9cb7fe640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15ba90 0x7fe9cc15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9ca7fc640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc0d0ce0 0x7fe9cc141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 -- 172.21.15.106:0/4241546723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15ba90 msgr2=0x7fe9cc15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15ba90 0x7fe9cc15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 -- 172.21.15.106:0/4241546723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc0d0ce0 msgr2=0x7fe9cc141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc0d0ce0 0x7fe9cc141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 -- 172.21.15.106:0/4241546723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc140e90 con 0x7fe9cc0ce850 2024-04-25T11:20:19.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9caffd640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc0ce850 0x7fe9cc15b410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc019070 tx=0x7fe9bc019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9c37fe640 1 -- 172.21.15.106:0/4241546723 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0ba070 con 0x7fe9cc0ce850 2024-04-25T11:20:19.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9c37fe640 1 -- 172.21.15.106:0/4241546723 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9bc0b43f0 con 0x7fe9cc0ce850 2024-04-25T11:20:19.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc15c3e0 con 0x7fe9cc0ce850 2024-04-25T11:20:19.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9c37fe640 1 -- 172.21.15.106:0/4241546723 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0b0040 con 0x7fe9cc0ce850 2024-04-25T11:20:19.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.091+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc13f5f0 con 0x7fe9cc0ce850 2024-04-25T11:20:19.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.095+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9cc0008e0 con 0x7fe9cc0ce850 2024-04-25T11:20:19.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.095+0000 7fe9c37fe640 1 -- 172.21.15.106:0/4241546723 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe9bc0bd070 con 0x7fe9cc0ce850 2024-04-25T11:20:19.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.095+0000 7fe9c37fe640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe9ac06f400 0x7fe9ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.095+0000 7fe9c37fe640 1 -- 172.21.15.106:0/4241546723 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe9bc128650 con 0x7fe9cc0ce850 2024-04-25T11:20:19.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.095+0000 7fe9ca7fc640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe9ac06f400 0x7fe9ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.095+0000 7fe9ca7fc640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe9ac06f400 0x7fe9ac071910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc143e60 tx=0x7fe9b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.099+0000 7fe9c37fe640 1 -- 172.21.15.106:0/4241546723 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe9bc0c4080 con 0x7fe9cc0ce850 2024-04-25T11:20:19.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.279+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9cc140690 con 0x7fe9cc0ce850 2024-04-25T11:20:19.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.279+0000 7fe9c37fe640 1 -- 172.21.15.106:0/4241546723 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fe9cc15c3e0 con 0x7fe9cc0ce850 2024-04-25T11:20:19.280 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:19.280 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:26.806158+0000","modified":"2024-04-25T11:19:29.823480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:20:19.280 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.279+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe9ac06f400 msgr2=0x7fe9ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.279+0000 7fe9d151d640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe9ac06f400 0x7fe9ac071910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc143e60 tx=0x7fe9b4009040 comp rx=0 tx=0).stop 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc0ce850 msgr2=0x7fe9cc15b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc0ce850 0x7fe9cc15b410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc019070 tx=0x7fe9bc019200 comp rx=0 tx=0).stop 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 shutdown_connections 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9cc15ba90 0x7fe9cc15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc0d0ce0 0x7fe9cc141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe9ac06f400 0x7fe9ac071910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 --2- 172.21.15.106:0/4241546723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc0ce850 0x7fe9cc15b410 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 >> 172.21.15.106:0/4241546723 conn(0x7fe9cc06c7c0 msgr2=0x7fe9cc0b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:19.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 shutdown_connections 2024-04-25T11:20:19.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.283+0000 7fe9d151d640 1 -- 172.21.15.106:0/4241546723 wait complete. 2024-04-25T11:20:19.300 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:19.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbbaadf9640 1 Processor -- start 2024-04-25T11:20:19.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbbaadf9640 1 -- start start 2024-04-25T11:20:19.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbbaadf9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40cf910 0x7fbba40d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbbaadf9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbba40b2e90 0x7fbba40b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbbaadf9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbba40cf150 con 0x7fbba40b37e0 2024-04-25T11:20:19.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbbaadf9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fbba40cfd20 con 0x7fbba40b2e90 2024-04-25T11:20:19.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbbaadf9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbba40b2730 con 0x7fbba40cf910 2024-04-25T11:20:19.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbba40b2e90 0x7fbba40b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba9396640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fbba40b37e0 0x7fbba4161f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:54982/0 (socket says 172.21.15.106:54982) 2024-04-25T11:20:19.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba9396640 1 -- 172.21.15.106:0/2703890126 learned_addr learned my addr 172.21.15.106:0/2703890126 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:19.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba8b95640 1 --2- 172.21.15.106:0/2703890126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40cf910 0x7fbba40d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba37fe640 1 -- 172.21.15.106:0/2703890126 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 29086763 0 0) 0x7fbba40cf150 con 0x7fbba40b37e0 2024-04-25T11:20:19.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba37fe640 1 -- 172.21.15.106:0/2703890126 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb8c003530 con 0x7fbba40b37e0 2024-04-25T11:20:19.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba8b95640 1 -- 172.21.15.106:0/2703890126 >> v2:172.21.15.134:3300/0 conn(0x7fbba40b2e90 msgr2=0x7fbba40b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:19.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba8b95640 1 --2- 172.21.15.106:0/2703890126 >> v2:172.21.15.134:3300/0 conn(0x7fbba40b2e90 0x7fbba40b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba8b95640 1 -- 172.21.15.106:0/2703890126 >> v1:172.21.15.134:6789/0 conn(0x7fbba40b37e0 legacy=0x7fbba4161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.447+0000 7fbba8b95640 1 -- 172.21.15.106:0/2703890126 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba40d0c70 con 0x7fbba40cf910 2024-04-25T11:20:19.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbba8b95640 1 --2- 172.21.15.106:0/2703890126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40cf910 0x7fbba40d1e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb980097a0 tx=0x7fbb9801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403eb13fa6d4035a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbba37fe640 1 -- 172.21.15.106:0/2703890126 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb980b0070 con 0x7fbba40cf910 2024-04-25T11:20:19.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbba37fe640 1 -- 172.21.15.106:0/2703890126 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb980ab600 con 0x7fbba40cf910 2024-04-25T11:20:19.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbba37fe640 1 -- 172.21.15.106:0/2703890126 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb980ab9b0 con 0x7fbba40cf910 2024-04-25T11:20:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbbaadf9640 1 -- 172.21.15.106:0/2703890126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40cf910 msgr2=0x7fbba40d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/2703890126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40cf910 0x7fbba40d1e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb980097a0 tx=0x7fbb9801c480 comp rx=0 tx=0).stop 2024-04-25T11:20:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbbaadf9640 1 -- 172.21.15.106:0/2703890126 shutdown_connections 2024-04-25T11:20:19.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/2703890126 >> v2:172.21.15.134:3300/0 conn(0x7fbba40b2e90 0x7fbba40b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/2703890126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40cf910 0x7fbba40d1e30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbbaadf9640 1 -- 172.21.15.106:0/2703890126 >> 172.21.15.106:0/2703890126 conn(0x7fbba40cffb0 msgr2=0x7fbba40d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:19.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbbaadf9640 1 -- 172.21.15.106:0/2703890126 shutdown_connections 2024-04-25T11:20:19.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.451+0000 7fbbaadf9640 1 -- 172.21.15.106:0/2703890126 wait complete. 2024-04-25T11:20:19.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 Processor -- start 2024-04-25T11:20:19.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 -- start start 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40b2e90 0x7fbba4159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbba40b37e0 0x7fbba415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba40cf910 0x7fbba415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbba40b8eb0 con 0x7fbba40b37e0 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbba40b8960 con 0x7fbba40cf910 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbba40b9030 con 0x7fbba40b2e90 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbba40b37e0 0x7fbba415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbba40b37e0 0x7fbba415e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59314/0 (socket says 172.21.15.106:59314) 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 -- 172.21.15.106:0/3077181401 learned_addr learned my addr 172.21.15.106:0/3077181401 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba8b95640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40b2e90 0x7fbba4159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba9396640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba40cf910 0x7fbba415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 -- 172.21.15.106:0/3077181401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40b2e90 msgr2=0x7fbba4159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40b2e90 0x7fbba4159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 -- 172.21.15.106:0/3077181401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba40cf910 msgr2=0x7fbba415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba40cf910 0x7fbba415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 -- 172.21.15.106:0/3077181401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba40ca930 con 0x7fbba40b37e0 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba3fff640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbba40b37e0 0x7fbba415e3f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400e8a0 tx=0x7fbb9400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba17fa640 1 -- 172.21.15.106:0/3077181401 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94017070 con 0x7fbba40b37e0 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba17fa640 1 -- 172.21.15.106:0/3077181401 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9400c940 con 0x7fbba40b37e0 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba40cac10 con 0x7fbba40b37e0 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbba17fa640 1 -- 172.21.15.106:0/3077181401 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9400ccf0 con 0x7fbba40b37e0 2024-04-25T11:20:19.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.455+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba40cb1d0 con 0x7fbba40b37e0 2024-04-25T11:20:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.459+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba4069d40 con 0x7fbba40b37e0 2024-04-25T11:20:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.459+0000 7fbba17fa640 1 -- 172.21.15.106:0/3077181401 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbb9401a070 con 0x7fbba40b37e0 2024-04-25T11:20:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.459+0000 7fbba17fa640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbb9006f400 0x7fbb90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.459+0000 7fbba8b95640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbb9006f400 0x7fbb90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.459+0000 7fbba17fa640 1 -- 172.21.15.106:0/3077181401 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbb9408f750 con 0x7fbba40b37e0 2024-04-25T11:20:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.459+0000 7fbba8b95640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbb9006f400 0x7fbb90071910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbb980ad000 tx=0x7fbb980a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.463+0000 7fbba17fa640 1 -- 172.21.15.106:0/3077181401 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbb9405c4a0 con 0x7fbba40b37e0 2024-04-25T11:20:19.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.619+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbba4069a10 con 0x7fbba40b37e0 2024-04-25T11:20:19.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbba17fa640 1 -- 172.21.15.106:0/3077181401 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16834 (secure 0 0 0) 0x7fbb9405bda0 con 0x7fbba40b37e0 2024-04-25T11:20:19.622 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:19.623 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:07.237284+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:19.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbb9006f400 msgr2=0x7fbb90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbb9006f400 0x7fbb90071910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbb980ad000 tx=0x7fbb980a6000 comp rx=0 tx=0).stop 2024-04-25T11:20:19.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbba40b37e0 msgr2=0x7fbba415e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbba40b37e0 0x7fbba415e3f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400e8a0 tx=0x7fbb9400eca0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 shutdown_connections 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba40cf910 0x7fbba415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbb9006f400 0x7fbb90071910 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbba40b37e0 0x7fbba415e3f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 --2- 172.21.15.106:0/3077181401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbba40b2e90 0x7fbba4159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 >> 172.21.15.106:0/3077181401 conn(0x7fbba40cffb0 msgr2=0x7fbba4071490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.623+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 shutdown_connections 2024-04-25T11:20:19.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.627+0000 7fbbaadf9640 1 -- 172.21.15.106:0/3077181401 wait complete. 2024-04-25T11:20:19.644 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:20:19.644 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:19.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a73561640 1 Processor -- start 2024-04-25T11:20:19.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a73561640 1 -- start start 2024-04-25T11:20:19.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a73561640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3a6c15e0a0 0x7f3a6c073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a73561640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a6c0ce050 con 0x7f3a6c0d0ce0 2024-04-25T11:20:19.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a73561640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a6c15e6b0 con 0x7f3a6c0ce850 2024-04-25T11:20:19.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a73561640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6c15e990 con 0x7f3a6c15e0a0 2024-04-25T11:20:19.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a712fd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3a6c15e0a0 0x7f3a6c073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a71afe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3a6c0d0ce0 0x7f3a6c0d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49094/0 (socket says 172.21.15.106:49094) 2024-04-25T11:20:19.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a71afe640 1 -- 172.21.15.106:0/3544188065 learned_addr learned my addr 172.21.15.106:0/3544188065 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:19.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a63fff640 1 -- 172.21.15.106:0/3544188065 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 947582970 0 0) 0x7f3a6c0ce050 con 0x7f3a6c0d0ce0 2024-04-25T11:20:19.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a63fff640 1 -- 172.21.15.106:0/3544188065 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a58003530 con 0x7f3a6c0d0ce0 2024-04-25T11:20:19.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a712fd640 1 -- 172.21.15.106:0/3544188065 >> v1:172.21.15.134:6789/0 conn(0x7f3a6c0ce850 legacy=0x7f3a6c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a712fd640 1 -- 172.21.15.106:0/3544188065 >> v1:172.21.15.106:6789/0 conn(0x7f3a6c0d0ce0 legacy=0x7f3a6c0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.787+0000 7f3a712fd640 1 -- 172.21.15.106:0/3544188065 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c15ec70 con 0x7f3a6c15e0a0 2024-04-25T11:20:19.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a712fd640 1 --2- 172.21.15.106:0/3544188065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c15e0a0 0x7f3a6c073460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3a680097c0 tx=0x7f3a6801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ca3d384bcaa54b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a63fff640 1 -- 172.21.15.106:0/3544188065 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a680b0070 con 0x7f3a6c15e0a0 2024-04-25T11:20:19.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a63fff640 1 -- 172.21.15.106:0/3544188065 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a68004230 con 0x7f3a6c15e0a0 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a63fff640 1 -- 172.21.15.106:0/3544188065 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68003620 con 0x7f3a6c15e0a0 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a73561640 1 -- 172.21.15.106:0/3544188065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c15e0a0 msgr2=0x7f3a6c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a73561640 1 --2- 172.21.15.106:0/3544188065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c15e0a0 0x7f3a6c073460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3a680097c0 tx=0x7f3a6801c4a0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a73561640 1 -- 172.21.15.106:0/3544188065 shutdown_connections 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a73561640 1 --2- 172.21.15.106:0/3544188065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c15e0a0 0x7f3a6c073460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a73561640 1 -- 172.21.15.106:0/3544188065 >> 172.21.15.106:0/3544188065 conn(0x7f3a6c06c7c0 msgr2=0x7f3a6c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a73561640 1 -- 172.21.15.106:0/3544188065 shutdown_connections 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.791+0000 7f3a73561640 1 -- 172.21.15.106:0/3544188065 wait complete. 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 Processor -- start 2024-04-25T11:20:19.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 -- start start 2024-04-25T11:20:19.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c0ce850 0x7f3a6c159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a6c0d0ce0 0x7f3a6c15d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a6c15e0a0 0x7f3a6c13a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c15f3f0 con 0x7f3a6c0ce850 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c15f270 con 0x7f3a6c0d0ce0 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c16bf10 con 0x7f3a6c15e0a0 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c0ce850 0x7f3a6c159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c0ce850 0x7f3a6c159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59330/0 (socket says 172.21.15.106:59330) 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 -- 172.21.15.106:0/993120624 learned_addr learned my addr 172.21.15.106:0/993120624 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a70afc640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a6c0d0ce0 0x7f3a6c15d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 -- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a6c15e0a0 msgr2=0x7f3a6c13a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a71afe640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a6c15e0a0 0x7f3a6c13a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a6c15e0a0 0x7f3a6c13a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 -- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a6c0d0ce0 msgr2=0x7f3a6c15d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a6c0d0ce0 0x7f3a6c15d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 -- 172.21.15.106:0/993120624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c15a650 con 0x7f3a6c0ce850 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a70afc640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a6c0d0ce0 0x7f3a6c15d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a712fd640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c0ce850 0x7f3a6c159b40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a680ae040 tx=0x7f3a68019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a61ffb640 1 -- 172.21.15.106:0/993120624 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a680b0040 con 0x7f3a6c0ce850 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a61ffb640 1 -- 172.21.15.106:0/993120624 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a680bc070 con 0x7f3a6c0ce850 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6c13f750 con 0x7f3a6c0ce850 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a61ffb640 1 -- 172.21.15.106:0/993120624 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a680043e0 con 0x7f3a6c0ce850 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6c13fd10 con 0x7f3a6c0ce850 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.795+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a6c13dc10 con 0x7f3a6c0ce850 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.799+0000 7f3a61ffb640 1 -- 172.21.15.106:0/993120624 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3a680b3070 con 0x7f3a6c0ce850 2024-04-25T11:20:19.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.799+0000 7f3a61ffb640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a5006f430 0x7f3a50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.799+0000 7f3a70afc640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a5006f430 0x7f3a50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.799+0000 7f3a61ffb640 1 -- 172.21.15.106:0/993120624 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3a68128e90 con 0x7f3a6c0ce850 2024-04-25T11:20:19.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.799+0000 7f3a70afc640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a5006f430 0x7f3a50071940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0d11d0 tx=0x7f3a5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.803+0000 7f3a61ffb640 1 -- 172.21.15.106:0/993120624 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3a680c6020 con 0x7f3a6c0ce850 2024-04-25T11:20:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.983+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a6c0cd5e0 con 0x7f3a6c0ce850 2024-04-25T11:20:19.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.983+0000 7f3a61ffb640 1 -- 172.21.15.106:0/993120624 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f3a6c15a650 con 0x7f3a6c0ce850 2024-04-25T11:20:19.985 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:19.985 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:26.806158+0000","modified":"2024-04-25T11:19:29.823480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:20:19.985 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T11:20:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a5006f430 msgr2=0x7f3a50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a5006f430 0x7f3a50071940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0d11d0 tx=0x7f3a5c009040 comp rx=0 tx=0).stop 2024-04-25T11:20:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c0ce850 msgr2=0x7f3a6c159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c0ce850 0x7f3a6c159b40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a680ae040 tx=0x7f3a68019240 comp rx=0 tx=0).stop 2024-04-25T11:20:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 shutdown_connections 2024-04-25T11:20:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a6c15e0a0 0x7f3a6c13a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a6c0d0ce0 0x7f3a6c15d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f3a5006f430 0x7f3a50071940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 --2- 172.21.15.106:0/993120624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c0ce850 0x7f3a6c159b40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 >> 172.21.15.106:0/993120624 conn(0x7f3a6c06c7c0 msgr2=0x7f3a6c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:19.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 shutdown_connections 2024-04-25T11:20:19.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:19.987+0000 7f3a73561640 1 -- 172.21.15.106:0/993120624 wait complete. 2024-04-25T11:20:20.006 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:20.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25ea73e640 1 Processor -- start 2024-04-25T11:20:20.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25ea73e640 1 -- start start 2024-04-25T11:20:20.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25ea73e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25e40cef60 0x7f25e4073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25ea73e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e40ce050 con 0x7f25e40d0ce0 2024-04-25T11:20:20.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25ea73e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e415e6b0 con 0x7f25e415e0a0 2024-04-25T11:20:20.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25ea73e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f25e415e990 con 0x7f25e40cef60 2024-04-25T11:20:20.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25db7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25e40cef60 0x7f25e4073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25e8cdb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f25e40d0ce0 0x7f25e40d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49102/0 (socket says 172.21.15.106:49102) 2024-04-25T11:20:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25e8cdb640 1 -- 172.21.15.106:0/2761525102 learned_addr learned my addr 172.21.15.106:0/2761525102 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25db7fe640 1 -- 172.21.15.106:0/2761525102 >> v1:172.21.15.134:6789/0 conn(0x7f25e415e0a0 legacy=0x7f25e40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25db7fe640 1 -- 172.21.15.106:0/2761525102 >> v1:172.21.15.106:6789/0 conn(0x7f25e40d0ce0 legacy=0x7f25e40d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.147+0000 7f25db7fe640 1 -- 172.21.15.106:0/2761525102 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e415ec70 con 0x7f25e40cef60 2024-04-25T11:20:20.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25db7fe640 1 --2- 172.21.15.106:0/2761525102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40cef60 0x7f25e4073460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0097c0 tx=0x7f25cc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd34f17008e645d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25e37fe640 1 -- 172.21.15.106:0/2761525102 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0b1070 con 0x7f25e40cef60 2024-04-25T11:20:20.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25e37fe640 1 -- 172.21.15.106:0/2761525102 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc0ac640 con 0x7f25e40cef60 2024-04-25T11:20:20.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25e37fe640 1 -- 172.21.15.106:0/2761525102 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0aca10 con 0x7f25e40cef60 2024-04-25T11:20:20.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 -- 172.21.15.106:0/2761525102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40cef60 msgr2=0x7f25e4073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 --2- 172.21.15.106:0/2761525102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40cef60 0x7f25e4073460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0097c0 tx=0x7f25cc01c460 comp rx=0 tx=0).stop 2024-04-25T11:20:20.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 -- 172.21.15.106:0/2761525102 shutdown_connections 2024-04-25T11:20:20.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 --2- 172.21.15.106:0/2761525102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40cef60 0x7f25e4073460 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 -- 172.21.15.106:0/2761525102 >> 172.21.15.106:0/2761525102 conn(0x7f25e406c7c0 msgr2=0x7f25e406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 -- 172.21.15.106:0/2761525102 shutdown_connections 2024-04-25T11:20:20.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 -- 172.21.15.106:0/2761525102 wait complete. 2024-04-25T11:20:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 Processor -- start 2024-04-25T11:20:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.151+0000 7f25ea73e640 1 -- start start 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40d0ce0 0x7f25e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e415e0a0 0x7f25e415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25e4136710 0x7f25e413b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25e415f3f0 con 0x7f25e40d0ce0 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25e415f270 con 0x7f25e415e0a0 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f25e416bf10 con 0x7f25e4136710 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40d0ce0 0x7f25e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40d0ce0 0x7f25e40b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59344/0 (socket says 172.21.15.106:59344) 2024-04-25T11:20:20.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 -- 172.21.15.106:0/2388650721 learned_addr learned my addr 172.21.15.106:0/2388650721 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 -- 172.21.15.106:0/2388650721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25e4136710 msgr2=0x7f25e413b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25e4136710 0x7f25e413b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 -- 172.21.15.106:0/2388650721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e415e0a0 msgr2=0x7f25e415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e415e0a0 0x7f25e415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 -- 172.21.15.106:0/2388650721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e41445a0 con 0x7f25e40d0ce0 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e3fff640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40d0ce0 0x7f25e40b4f00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f25d400b410 tx=0x7f25d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e17fa640 1 -- 172.21.15.106:0/2388650721 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4015030 con 0x7f25e40d0ce0 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e17fa640 1 -- 172.21.15.106:0/2388650721 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d4004020 con 0x7f25e40d0ce0 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e40b5750 con 0x7f25e40d0ce0 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e17fa640 1 -- 172.21.15.106:0/2388650721 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4004410 con 0x7f25e40d0ce0 2024-04-25T11:20:20.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e4144810 con 0x7f25e40d0ce0 2024-04-25T11:20:20.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e41430d0 con 0x7f25e40d0ce0 2024-04-25T11:20:20.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e17fa640 1 -- 172.21.15.106:0/2388650721 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f25d4018070 con 0x7f25e40d0ce0 2024-04-25T11:20:20.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e17fa640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f25c406f400 0x7f25c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25e17fa640 1 -- 172.21.15.106:0/2388650721 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f25d408f690 con 0x7f25e40d0ce0 2024-04-25T11:20:20.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.155+0000 7f25db7fe640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f25c406f400 0x7f25c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.159+0000 7f25db7fe640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f25c406f400 0x7f25c4071910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0ae000 tx=0x7f25cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.159+0000 7f25e17fa640 1 -- 172.21.15.106:0/2388650721 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25d405bf90 con 0x7f25e40d0ce0 2024-04-25T11:20:20.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25e4142df0 con 0x7f25e40d0ce0 2024-04-25T11:20:20.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25e17fa640 1 -- 172.21.15.106:0/2388650721 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f25d4060290 con 0x7f25e40d0ce0 2024-04-25T11:20:20.359 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:20.359 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/26175976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":26175976},{"type":"v1","addr":"172.21.15.106:6835","nonce":26175976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/266925120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":266925120},{"type":"v1","addr":"172.21.15.106:6837","nonce":266925120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3980761123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980761123},{"type":"v1","addr":"172.21.15.134:6835","nonce":3980761123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:26.806158+0000","modified":"2024-04-25T11:19:29.823480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6833/3507437799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3507437799},{"type":"v1","addr":"172.21.15.134:6833","nonce":3507437799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:20:20.359 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T11:20:20.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f25c406f400 msgr2=0x7f25c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f25c406f400 0x7f25c4071910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0ae000 tx=0x7f25cc008040 comp rx=0 tx=0).stop 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40d0ce0 msgr2=0x7f25e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40d0ce0 0x7f25e40b4f00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f25d400b410 tx=0x7f25d400b8d0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 shutdown_connections 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25e4136710 0x7f25e413b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e415e0a0 0x7f25e415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f25c406f400 0x7f25c4071910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 --2- 172.21.15.106:0/2388650721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e40d0ce0 0x7f25e40b4f00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.359+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 >> 172.21.15.106:0/2388650721 conn(0x7f25e406c7c0 msgr2=0x7f25e40b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.363+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 shutdown_connections 2024-04-25T11:20:20.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.363+0000 7f25ea73e640 1 -- 172.21.15.106:0/2388650721 wait complete. 2024-04-25T11:20:20.380 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:20.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.511+0000 7f0435217640 1 Processor -- start 2024-04-25T11:20:20.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.511+0000 7f0435217640 1 -- start start 2024-04-25T11:20:20.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.511+0000 7f0435217640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04300ba970 0x7f04300d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.511+0000 7f0435217640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f04300d2160 0x7f04300d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.511+0000 7f0435217640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04300b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.511+0000 7f0435217640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f04300d1100 con 0x7f04300d2160 2024-04-25T11:20:20.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.511+0000 7f0435217640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f04300d2ab0 con 0x7f04300ba970 2024-04-25T11:20:20.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.511+0000 7f0435217640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04300d0530 con 0x7f04300b5120 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f04300d2160 0x7f04300d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f04300d2160 0x7f04300d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59354/0 (socket says 172.21.15.106:59354) 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 -- 172.21.15.106:0/1010805123 learned_addr learned my addr 172.21.15.106:0/1010805123 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042ed9d640 1 --2- 172.21.15.106:0/1010805123 >> v2:172.21.15.134:3300/0 conn(0x7f04300ba970 0x7f04300d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042f59e640 1 --2- 172.21.15.106:0/1010805123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04300b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 -- 172.21.15.106:0/1010805123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 msgr2=0x7f04300b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 --2- 172.21.15.106:0/1010805123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04300b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 -- 172.21.15.106:0/1010805123 >> v2:172.21.15.134:3300/0 conn(0x7f04300ba970 msgr2=0x7f04300d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 --2- 172.21.15.106:0/1010805123 >> v2:172.21.15.134:3300/0 conn(0x7f04300ba970 0x7f04300d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 -- 172.21.15.106:0/1010805123 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04300d07d0 con 0x7f04300d2160 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042ed9d640 1 --2- 172.21.15.106:0/1010805123 >> v2:172.21.15.134:3300/0 conn(0x7f04300ba970 0x7f04300d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:20.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042f59e640 1 --2- 172.21.15.106:0/1010805123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04300b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042e59c640 1 --2- 172.21.15.106:0/1010805123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300d2570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f043008e700 tx=0x7f042401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f768f7da3b3044a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042dd9b640 1 -- 172.21.15.106:0/1010805123 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240b0070 con 0x7f04300d2160 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042dd9b640 1 -- 172.21.15.106:0/1010805123 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04240ab600 con 0x7f04300d2160 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f042dd9b640 1 -- 172.21.15.106:0/1010805123 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240ab9d0 con 0x7f04300d2160 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 -- 172.21.15.106:0/1010805123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 msgr2=0x7f04300d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 --2- 172.21.15.106:0/1010805123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300d2570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f043008e700 tx=0x7f042401c520 comp rx=0 tx=0).stop 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 -- 172.21.15.106:0/1010805123 shutdown_connections 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 --2- 172.21.15.106:0/1010805123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04300b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 --2- 172.21.15.106:0/1010805123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300d2570 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 --2- 172.21.15.106:0/1010805123 >> v2:172.21.15.134:3300/0 conn(0x7f04300ba970 0x7f04300d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 -- 172.21.15.106:0/1010805123 >> 172.21.15.106:0/1010805123 conn(0x7f04300d57e0 msgr2=0x7f04300d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 -- 172.21.15.106:0/1010805123 shutdown_connections 2024-04-25T11:20:20.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.515+0000 7f0435217640 1 -- 172.21.15.106:0/1010805123 wait complete. 2024-04-25T11:20:20.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 Processor -- start 2024-04-25T11:20:20.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 -- start start 2024-04-25T11:20:20.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04301589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04300ba970 0x7f04300cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04300b4da0 con 0x7f04300d2160 2024-04-25T11:20:20.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04301605b0 con 0x7f04300ba970 2024-04-25T11:20:20.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04300b4f20 con 0x7f04300b5120 2024-04-25T11:20:20.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59356/0 (socket says 172.21.15.106:59356) 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 -- 172.21.15.106:0/3027974874 learned_addr learned my addr 172.21.15.106:0/3027974874 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042e59c640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04300ba970 0x7f04300cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042ed9d640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04301589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 -- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 msgr2=0x7f04301589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04301589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 -- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04300ba970 msgr2=0x7f04300cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04300ba970 0x7f04300cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 -- 172.21.15.106:0/3027974874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04300ca8e0 con 0x7f04300d2160 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042ed9d640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04301589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f042f59e640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300ce2f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f042800cca0 tx=0x7f042800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f04177fe640 1 -- 172.21.15.106:0/3027974874 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042800f680 con 0x7f04300d2160 2024-04-25T11:20:20.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f04177fe640 1 -- 172.21.15.106:0/3027974874 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042800f840 con 0x7f04300d2160 2024-04-25T11:20:20.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04300cabc0 con 0x7f04300d2160 2024-04-25T11:20:20.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f04177fe640 1 -- 172.21.15.106:0/3027974874 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042800fc30 con 0x7f04300d2160 2024-04-25T11:20:20.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.519+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0430160ab0 con 0x7f04300d2160 2024-04-25T11:20:20.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.523+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043006e1d0 con 0x7f04300d2160 2024-04-25T11:20:20.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.523+0000 7f04177fe640 1 -- 172.21.15.106:0/3027974874 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04280213f0 con 0x7f04300d2160 2024-04-25T11:20:20.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.523+0000 7f04177fe640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f041c06f430 0x7f041c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.523+0000 7f042ed9d640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f041c06f430 0x7f041c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.523+0000 7f04177fe640 1 -- 172.21.15.106:0/3027974874 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0428090200 con 0x7f04300d2160 2024-04-25T11:20:20.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.523+0000 7f042ed9d640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f041c06f430 0x7f041c071940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0420006fd0 tx=0x7f0420008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.527+0000 7f04177fe640 1 -- 172.21.15.106:0/3027974874 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0428014030 con 0x7f04300d2160 2024-04-25T11:20:20.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.687+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04300b4980 con 0x7f04300d2160 2024-04-25T11:20:20.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.691+0000 7f04177fe640 1 -- 172.21.15.106:0/3027974874 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16834 (secure 0 0 0) 0x7f042805d1e0 con 0x7f04300d2160 2024-04-25T11:20:20.691 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:20.691 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:07.237284+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:24.592862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:25.761448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:20.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f041c06f430 msgr2=0x7f041c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f041c06f430 0x7f041c071940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0420006fd0 tx=0x7f0420008040 comp rx=0 tx=0).stop 2024-04-25T11:20:20.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 msgr2=0x7f04300ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300ce2f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f042800cca0 tx=0x7f042800a650 comp rx=0 tx=0).stop 2024-04-25T11:20:20.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 shutdown_connections 2024-04-25T11:20:20.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f041c06f430 0x7f041c071940 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04300d2160 0x7f04300ce2f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04300ba970 0x7f04300cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 --2- 172.21.15.106:0/3027974874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04300b5120 0x7f04301589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 >> 172.21.15.106:0/3027974874 conn(0x7f04300d57e0 msgr2=0x7f04300b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 shutdown_connections 2024-04-25T11:20:20.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.695+0000 7f0435217640 1 -- 172.21.15.106:0/3027974874 wait complete. 2024-04-25T11:20:20.714 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:20:20.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.855+0000 7fee0df90640 1 Processor -- start 2024-04-25T11:20:20.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.855+0000 7fee0df90640 1 -- start start 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.855+0000 7fee0df90640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee080b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.855+0000 7fee0df90640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fee080d44a0 0x7fee080d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.855+0000 7fee0df90640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee080d1350 con 0x7fee080d2150 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.855+0000 7fee0df90640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fee080b9270 con 0x7fee080d44a0 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.855+0000 7fee0df90640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fee08163cc0 con 0x7fee0815fd10 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee06ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fee080d2150 0x7fee080d2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49108/0 (socket says 172.21.15.106:49108) 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee06ffd640 1 -- 172.21.15.106:0/3030673636 learned_addr learned my addr 172.21.15.106:0/3030673636 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee07fff640 1 --2- 172.21.15.106:0/3030673636 >> v2:172.21.15.106:3300/0 conn(0x7fee080d44a0 0x7fee080d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee077fe640 1 --2- 172.21.15.106:0/3030673636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee080b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee067fc640 1 -- 172.21.15.106:0/3030673636 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3275861267 0 0) 0x7fee080d1350 con 0x7fee080d2150 2024-04-25T11:20:20.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee067fc640 1 -- 172.21.15.106:0/3030673636 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedf4003530 con 0x7fee080d2150 2024-04-25T11:20:20.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee07fff640 1 -- 172.21.15.106:0/3030673636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 msgr2=0x7fee080b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee07fff640 1 --2- 172.21.15.106:0/3030673636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee080b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee07fff640 1 -- 172.21.15.106:0/3030673636 >> v1:172.21.15.106:6789/0 conn(0x7fee080d2150 legacy=0x7fee080d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee07fff640 1 -- 172.21.15.106:0/3030673636 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee08163fa0 con 0x7fee080d44a0 2024-04-25T11:20:20.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee07fff640 1 --2- 172.21.15.106:0/3030673636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee080d48b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee00009d90 tx=0x7fee0001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bb868a7c1121ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee067fc640 1 -- 172.21.15.106:0/3030673636 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000b2070 con 0x7fee080d44a0 2024-04-25T11:20:20.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee067fc640 1 -- 172.21.15.106:0/3030673636 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee000a5830 con 0x7fee080d44a0 2024-04-25T11:20:20.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.859+0000 7fee067fc640 1 -- 172.21.15.106:0/3030673636 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000a5c00 con 0x7fee080d44a0 2024-04-25T11:20:20.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- 172.21.15.106:0/3030673636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 msgr2=0x7fee080d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 --2- 172.21.15.106:0/3030673636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee080d48b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee00009d90 tx=0x7fee0001c970 comp rx=0 tx=0).stop 2024-04-25T11:20:20.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- 172.21.15.106:0/3030673636 shutdown_connections 2024-04-25T11:20:20.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 --2- 172.21.15.106:0/3030673636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee080d48b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 --2- 172.21.15.106:0/3030673636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee080b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- 172.21.15.106:0/3030673636 >> 172.21.15.106:0/3030673636 conn(0x7fee0806d7f0 msgr2=0x7fee0806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- 172.21.15.106:0/3030673636 shutdown_connections 2024-04-25T11:20:20.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- 172.21.15.106:0/3030673636 wait complete. 2024-04-25T11:20:20.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 Processor -- start 2024-04-25T11:20:20.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- start start 2024-04-25T11:20:20.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee080d2150 0x7fee08154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee08154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee08155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fee08163a60 con 0x7fee080d44a0 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fee081638e0 con 0x7fee080d2150 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee0df90640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fee080b49a0 con 0x7fee0815fd10 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee08154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee08154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59368/0 (socket says 172.21.15.106:59368) 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 -- 172.21.15.106:0/1902915987 learned_addr learned my addr 172.21.15.106:0/1902915987 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee07fff640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee08155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee077fe640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee080d2150 0x7fee08154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 -- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 msgr2=0x7fee08155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee08155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 -- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee080d2150 msgr2=0x7fee08154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee080d2150 0x7fee08154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 -- 172.21.15.106:0/1902915987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee08160460 con 0x7fee080d44a0 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee07fff640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee08155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee077fe640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee080d2150 0x7fee08154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.863+0000 7fee06ffd640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee08154c30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c3f0 tx=0x7fedf800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fedfffff640 1 -- 172.21.15.106:0/1902915987 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8017070 con 0x7fee080d44a0 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fedfffff640 1 -- 172.21.15.106:0/1902915987 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf8005cf0 con 0x7fee080d44a0 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee08160740 con 0x7fee080d44a0 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fedfffff640 1 -- 172.21.15.106:0/1902915987 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80072f0 con 0x7fee080d44a0 2024-04-25T11:20:20.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee08160d00 con 0x7fee080d44a0 2024-04-25T11:20:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee08050750 con 0x7fee080d44a0 2024-04-25T11:20:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fedfffff640 1 -- 172.21.15.106:0/1902915987 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fedf801a070 con 0x7fee080d44a0 2024-04-25T11:20:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fedfffff640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fede806f430 0x7fede8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fee077fe640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fede806f430 0x7fede8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fedfffff640 1 -- 172.21.15.106:0/1902915987 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fedf808eb20 con 0x7fee080d44a0 2024-04-25T11:20:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.867+0000 7fee077fe640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fede806f430 0x7fede8071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fedf00097a0 tx=0x7fedf0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:20.871+0000 7fedfffff640 1 -- 172.21.15.106:0/1902915987 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fedf8021020 con 0x7fee080d44a0 2024-04-25T11:20:21.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.051+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fee080d3be0 con 0x7fee080d44a0 2024-04-25T11:20:21.054 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-25T11:20:21.055+0000 7f8fc7653640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:20:21.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.055+0000 7fedfffff640 1 -- 172.21.15.106:0/1902915987 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fedf805ba70 con 0x7fee080d44a0 2024-04-25T11:20:21.057 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:20:21.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fede806f430 msgr2=0x7fede8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fede806f430 0x7fede8071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fedf00097a0 tx=0x7fedf0008040 comp rx=0 tx=0).stop 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 msgr2=0x7fee08154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee08154c30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c3f0 tx=0x7fedf800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 shutdown_connections 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee0815fd10 0x7fee08155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fede806f430 0x7fede8071940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee080d44a0 0x7fee08154c30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 --2- 172.21.15.106:0/1902915987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee080d2150 0x7fee08154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 >> 172.21.15.106:0/1902915987 conn(0x7fee0806d7f0 msgr2=0x7fee080b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 shutdown_connections 2024-04-25T11:20:21.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.059+0000 7fee0df90640 1 -- 172.21.15.106:0/1902915987 wait complete. 2024-04-25T11:20:21.071 INFO:tasks.ceph.mds.d.smithi134.stderr: -3473> 2024-04-25T11:19:30.938+0000 7f40a076d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:20:21.079 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:20:21.166 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-04-25T11:20:21.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.223+0000 7f5de368d640 1 Processor -- start 2024-04-25T11:20:21.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.223+0000 7f5de368d640 1 -- start start 2024-04-25T11:20:21.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de368d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ddc143fa0 0x7f5ddc0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de368d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0d1e50 0x7f5ddc0b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de368d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ddc073300 con 0x7f5ddc072a20 2024-04-25T11:20:21.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de368d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5ddc0735e0 con 0x7f5ddc143fa0 2024-04-25T11:20:21.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de368d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ddc0738c0 con 0x7f5ddc0d1e50 2024-04-25T11:20:21.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de1429640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ddc143fa0 0x7f5ddc0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de0c28640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5ddc072a20 0x7f5ddc0d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49118/0 (socket says 172.21.15.106:49118) 2024-04-25T11:20:21.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de0c28640 1 -- 172.21.15.106:0/2709899008 learned_addr learned my addr 172.21.15.106:0/2709899008 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:21.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de1c2a640 1 --2- 172.21.15.106:0/2709899008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0d1e50 0x7f5ddc0b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de1429640 1 -- 172.21.15.106:0/2709899008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0d1e50 msgr2=0x7f5ddc0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:21.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de1429640 1 --2- 172.21.15.106:0/2709899008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0d1e50 0x7f5ddc0b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de1429640 1 -- 172.21.15.106:0/2709899008 >> v1:172.21.15.106:6789/0 conn(0x7f5ddc072a20 legacy=0x7f5ddc0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5dd3fff640 1 -- 172.21.15.106:0/2709899008 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 24804072 0 0) 0x7f5ddc073300 con 0x7f5ddc072a20 2024-04-25T11:20:21.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de1429640 1 -- 172.21.15.106:0/2709899008 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc0b5040 con 0x7f5ddc143fa0 2024-04-25T11:20:21.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de1c2a640 1 --2- 172.21.15.106:0/2709899008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0d1e50 0x7f5ddc0b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:21.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.227+0000 7f5de1429640 1 --2- 172.21.15.106:0/2709899008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc143fa0 0x7f5ddc0cd930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80097c0 tx=0x7f5dd801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07e33d079022288 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:21.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5dd3fff640 1 -- 172.21.15.106:0/2709899008 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80aa5f0 con 0x7f5ddc143fa0 2024-04-25T11:20:21.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5dd3fff640 1 -- 172.21.15.106:0/2709899008 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd80af070 con 0x7f5ddc143fa0 2024-04-25T11:20:21.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5dd3fff640 1 -- 172.21.15.106:0/2709899008 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80aa9b0 con 0x7f5ddc143fa0 2024-04-25T11:20:21.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 -- 172.21.15.106:0/2709899008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc143fa0 msgr2=0x7f5ddc0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 --2- 172.21.15.106:0/2709899008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc143fa0 0x7f5ddc0cd930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80097c0 tx=0x7f5dd801c470 comp rx=0 tx=0).stop 2024-04-25T11:20:21.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 -- 172.21.15.106:0/2709899008 shutdown_connections 2024-04-25T11:20:21.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 --2- 172.21.15.106:0/2709899008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0d1e50 0x7f5ddc0b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 --2- 172.21.15.106:0/2709899008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc143fa0 0x7f5ddc0cd930 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 -- 172.21.15.106:0/2709899008 >> 172.21.15.106:0/2709899008 conn(0x7f5ddc06c9f0 msgr2=0x7f5ddc06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:21.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 -- 172.21.15.106:0/2709899008 shutdown_connections 2024-04-25T11:20:21.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 -- 172.21.15.106:0/2709899008 wait complete. 2024-04-25T11:20:21.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 Processor -- start 2024-04-25T11:20:21.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 -- start start 2024-04-25T11:20:21.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc072a20 0x7f5ddc0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.231+0000 7f5de368d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc143fa0 0x7f5ddc160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de368d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc160390 0x7f5ddc152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de368d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc16ad20 con 0x7f5ddc160390 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de368d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc16aba0 con 0x7f5ddc072a20 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de368d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ddc16aea0 con 0x7f5ddc143fa0 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc160390 0x7f5ddc152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc160390 0x7f5ddc152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59390/0 (socket says 172.21.15.106:59390) 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 -- 172.21.15.106:0/3697453793 learned_addr learned my addr 172.21.15.106:0/3697453793 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1429640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc072a20 0x7f5ddc0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de0c28640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc143fa0 0x7f5ddc160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 -- 172.21.15.106:0/3697453793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc143fa0 msgr2=0x7f5ddc160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc143fa0 0x7f5ddc160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 -- 172.21.15.106:0/3697453793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc072a20 msgr2=0x7f5ddc0b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc072a20 0x7f5ddc0b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 -- 172.21.15.106:0/3697453793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc151560 con 0x7f5ddc160390 2024-04-25T11:20:21.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1c2a640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc160390 0x7f5ddc152ef0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4007e10 tx=0x7f5dd400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:21.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5dd1ffb640 1 -- 172.21.15.106:0/3697453793 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd401a070 con 0x7f5ddc160390 2024-04-25T11:20:21.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5dd1ffb640 1 -- 172.21.15.106:0/3697453793 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd4003ba0 con 0x7f5ddc160390 2024-04-25T11:20:21.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc151840 con 0x7f5ddc160390 2024-04-25T11:20:21.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5dd1ffb640 1 -- 172.21.15.106:0/3697453793 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400de20 con 0x7f5ddc160390 2024-04-25T11:20:21.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc0b2e40 con 0x7f5ddc160390 2024-04-25T11:20:21.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ddc1512d0 con 0x7f5ddc160390 2024-04-25T11:20:21.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5dd1ffb640 1 -- 172.21.15.106:0/3697453793 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5dd401d070 con 0x7f5ddc160390 2024-04-25T11:20:21.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5dd1ffb640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5dc006f400 0x7f5dc0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1429640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5dc006f400 0x7f5dc0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5dd1ffb640 1 -- 172.21.15.106:0/3697453793 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f5dd408f4f0 con 0x7f5ddc160390 2024-04-25T11:20:21.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.235+0000 7f5de1429640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5dc006f400 0x7f5dc0071910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ae000 tx=0x7f5dd80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:21.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.243+0000 7f5dd1ffb640 1 -- 172.21.15.106:0/3697453793 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5dd4010070 con 0x7f5ddc160390 2024-04-25T11:20:21.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:21.431+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5ddc050750 con 0x7f5ddc160390 2024-04-25T11:20:22.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.059+0000 7f5dd1ffb640 1 -- 172.21.15.106:0/3697453793 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f5dd405c3a0 con 0x7f5ddc160390 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5dc006f400 msgr2=0x7f5dc0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5dc006f400 0x7f5dc0071910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ae000 tx=0x7f5dd80a7000 comp rx=0 tx=0).stop 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc160390 msgr2=0x7f5ddc152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc160390 0x7f5ddc152ef0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4007e10 tx=0x7f5dd400d4a0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 shutdown_connections 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5dc006f400 0x7f5dc0071910 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ddc160390 0x7f5ddc152ef0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc143fa0 0x7f5ddc160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 --2- 172.21.15.106:0/3697453793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc072a20 0x7f5ddc0b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 >> 172.21.15.106:0/3697453793 conn(0x7f5ddc06c9f0 msgr2=0x7f5ddc06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 shutdown_connections 2024-04-25T11:20:22.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.063+0000 7f5de368d640 1 -- 172.21.15.106:0/3697453793 wait complete. 2024-04-25T11:20:22.089 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:20:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.247+0000 7f0b27d93640 1 Processor -- start 2024-04-25T11:20:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.247+0000 7f0b27d93640 1 -- start start 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.247+0000 7f0b27d93640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201622c0 0x7f0b200c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.247+0000 7f0b27d93640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b20155150 con 0x7f0b200d0b10 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.247+0000 7f0b27d93640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2015e610 con 0x7f0b20154d40 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.247+0000 7f0b27d93640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2015e8f0 con 0x7f0b201622c0 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.247+0000 7f0b2532e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b200d0b10 0x7f0b200c69f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49128/0 (socket says 172.21.15.106:49128) 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.247+0000 7f0b2532e640 1 -- 172.21.15.106:0/218664763 learned_addr learned my addr 172.21.15.106:0/218664763 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733954626 0 0) 0x7f0b20155150 con 0x7f0b200d0b10 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b26330640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201622c0 0x7f0b200c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b0c003530 con 0x7f0b200d0b10 2024-04-25T11:20:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4191316424 0 0) 0x7f0b0c003530 con 0x7f0b200d0b10 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201622c0 msgr2=0x7f0b200c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201622c0 0x7f0b200c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 >> v1:172.21.15.134:6789/0 conn(0x7f0b20154d40 legacy=0x7f0b200d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2015ebd0 con 0x7f0b200d0b10 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f0b10001680 con 0x7f0b200d0b10 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 >> v1:172.21.15.106:6789/0 conn(0x7f0b200d0b10 legacy=0x7f0b200c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c0039f0 0x7f0b0c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b0c004810 0x7f0b0c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201622c0 0x7f0b0c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2015e8f0 con 0x7f0b0c004810 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c00bb00 con 0x7f0b0c0039f0 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0c00bd00 con 0x7f0b201622c0 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b2532e640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b0c004810 0x7f0b0c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b25b2f640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c0039f0 0x7f0b0c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b26330640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201622c0 0x7f0b0c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b2532e640 1 -- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201622c0 msgr2=0x7f0b0c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:22.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b2532e640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201622c0 0x7f0b0c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b2532e640 1 -- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c0039f0 msgr2=0x7f0b0c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:22.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b2532e640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c0039f0 0x7f0b0c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b2532e640 1 -- 172.21.15.106:0/218664763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c003530 con 0x7f0b0c004810 2024-04-25T11:20:22.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b2532e640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b0c004810 0x7f0b0c004c20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0b10001ec0 tx=0x7f0b1001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c971b5e2eaf51bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10007d40 con 0x7f0b0c004810 2024-04-25T11:20:22.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10005b00 con 0x7f0b0c004810 2024-04-25T11:20:22.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.251+0000 7f0b24b2d640 1 -- 172.21.15.106:0/218664763 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10007070 con 0x7f0b0c004810 2024-04-25T11:20:22.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- 172.21.15.106:0/218664763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b0c004810 msgr2=0x7f0b0c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 --2- 172.21.15.106:0/218664763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b0c004810 0x7f0b0c004c20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0b10001ec0 tx=0x7f0b1001f370 comp rx=0 tx=0).stop 2024-04-25T11:20:22.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b26330640 1 -- 172.21.15.106:0/218664763 reap_dead start 2024-04-25T11:20:22.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- 172.21.15.106:0/218664763 shutdown_connections 2024-04-25T11:20:22.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- 172.21.15.106:0/218664763 >> 172.21.15.106:0/218664763 conn(0x7f0b20068c40 msgr2=0x7f0b2006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:22.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- 172.21.15.106:0/218664763 shutdown_connections 2024-04-25T11:20:22.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- 172.21.15.106:0/218664763 wait complete. 2024-04-25T11:20:22.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 Processor -- start 2024-04-25T11:20:22.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- start start 2024-04-25T11:20:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c004810 0x7f0b20156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201e7560 0x7f0b201f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2006f040 0x7f0b2006f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201626d0 con 0x7f0b2006f040 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2015f190 con 0x7f0b0c004810 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b27d93640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2015e2c0 con 0x7f0b201e7560 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b26330640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2006f040 0x7f0b2006f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b26330640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2006f040 0x7f0b2006f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59396/0 (socket says 172.21.15.106:59396) 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.255+0000 7f0b26330640 1 -- 172.21.15.106:0/4174086010 learned_addr learned my addr 172.21.15.106:0/4174086010 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b25b2f640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c004810 0x7f0b20156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b2532e640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201e7560 0x7f0b201f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b26330640 1 -- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201e7560 msgr2=0x7f0b201f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b26330640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201e7560 0x7f0b201f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b26330640 1 -- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c004810 msgr2=0x7f0b20156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b26330640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c004810 0x7f0b20156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b26330640 1 -- 172.21.15.106:0/4174086010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2006f990 con 0x7f0b2006f040 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b25b2f640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c004810 0x7f0b20156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b2532e640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201e7560 0x7f0b201f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b26330640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2006f040 0x7f0b2006f450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0b200c7db0 tx=0x7f0b180115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b167fc640 1 -- 172.21.15.106:0/4174086010 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b180039a0 con 0x7f0b2006f040 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b167fc640 1 -- 172.21.15.106:0/4174086010 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1801b070 con 0x7f0b2006f040 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b201e7d40 con 0x7f0b2006f040 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b167fc640 1 -- 172.21.15.106:0/4174086010 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18003ba0 con 0x7f0b2006f040 2024-04-25T11:20:22.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201e8300 con 0x7f0b2006f040 2024-04-25T11:20:22.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b20074520 con 0x7f0b2006f040 2024-04-25T11:20:22.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b167fc640 1 -- 172.21.15.106:0/4174086010 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0b18004030 con 0x7f0b2006f040 2024-04-25T11:20:22.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b167fc640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0b0406f400 0x7f0b04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b25b2f640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0b0406f400 0x7f0b04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b167fc640 1 -- 172.21.15.106:0/4174086010 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f0b1808faa0 con 0x7f0b2006f040 2024-04-25T11:20:22.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.259+0000 7f0b25b2f640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0b0406f400 0x7f0b04071910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b200b6e70 tx=0x7f0b1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.263+0000 7f0b167fc640 1 -- 172.21.15.106:0/4174086010 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0b2006f990 con 0x7f0b2006f040 2024-04-25T11:20:22.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:22.415+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0b201e7ad0 con 0x7f0b2006f040 2024-04-25T11:20:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.343+0000 7f0b167fc640 1 -- 172.21.15.106:0/4174086010 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f0b1805c9e0 con 0x7f0b2006f040 2024-04-25T11:20:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:20:23.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0b0406f400 msgr2=0x7f0b04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0b0406f400 0x7f0b04071910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b200b6e70 tx=0x7f0b1c009040 comp rx=0 tx=0).stop 2024-04-25T11:20:23.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2006f040 msgr2=0x7f0b2006f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2006f040 0x7f0b2006f450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0b200c7db0 tx=0x7f0b180115a0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 shutdown_connections 2024-04-25T11:20:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f0b0406f400 0x7f0b04071910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2006f040 0x7f0b2006f450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b201e7560 0x7f0b201f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 --2- 172.21.15.106:0/4174086010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b0c004810 0x7f0b20156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 >> 172.21.15.106:0/4174086010 conn(0x7f0b20068c40 msgr2=0x7f0b20153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:23.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 shutdown_connections 2024-04-25T11:20:23.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.347+0000 7f0b27d93640 1 -- 172.21.15.106:0/4174086010 wait complete. 2024-04-25T11:20:23.379 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:20:23.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70eb6e3640 1 Processor -- start 2024-04-25T11:20:23.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70eb6e3640 1 -- start start 2024-04-25T11:20:23.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70eb6e3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f70e40b6f50 0x7f70e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70eb6e3640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e40b1c90 con 0x7f70e40b0d40 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70eb6e3640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e40b0260 con 0x7f70e40b16f0 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70eb6e3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f70e40b0540 con 0x7f70e40b6f50 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70e8c7e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f70e40b0d40 0x7f70e40b6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49144/0 (socket says 172.21.15.106:49144) 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70e8c7e640 1 -- 172.21.15.106:0/2374509018 learned_addr learned my addr 172.21.15.106:0/2374509018 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70e9c80640 1 --2- 172.21.15.106:0/2374509018 >> v2:172.21.15.134:3300/0 conn(0x7f70e40b6f50 0x7f70e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640296999 0 0) 0x7f70e40b1c90 con 0x7f70e40b0d40 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70d0005740 con 0x7f70e40b0d40 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3145194431 0 0) 0x7f70d0005740 con 0x7f70e40b0d40 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 >> v2:172.21.15.134:3300/0 conn(0x7f70e40b6f50 msgr2=0x7f70e40b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 --2- 172.21.15.106:0/2374509018 >> v2:172.21.15.134:3300/0 conn(0x7f70e40b6f50 0x7f70e40b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 >> v1:172.21.15.134:6789/0 conn(0x7f70e40b16f0 legacy=0x7f70e40b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e40b0820 con 0x7f70e40b0d40 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f70e0001680 con 0x7f70e40b0d40 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 >> v1:172.21.15.106:6789/0 conn(0x7f70e40b0d40 legacy=0x7f70e40b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d0005bd0 0x7f70d0006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d0006a20 0x7f70d0006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d00073e0 0x7f70d000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40b0540 con 0x7f70d0006a20 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70d000e380 con 0x7f70d0005bd0 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.539+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f70d000e580 con 0x7f70d00073e0 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e8c7e640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d0006a20 0x7f70d0006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e947f640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d0005bd0 0x7f70d0006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e9c80640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d00073e0 0x7f70d000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e8c7e640 1 -- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d00073e0 msgr2=0x7f70d000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:23.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e8c7e640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d00073e0 0x7f70d000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e8c7e640 1 -- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d0005bd0 msgr2=0x7f70d0006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:23.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e8c7e640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d0005bd0 0x7f70d0006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e8c7e640 1 -- 172.21.15.106:0/2374509018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d0005740 con 0x7f70d0006a20 2024-04-25T11:20:23.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e9c80640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d00073e0 0x7f70d000dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:23.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e9c80640 1 -- 172.21.15.106:0/2374509018 reap_dead start 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70e8c7e640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d0006a20 0x7f70d0006e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f70e000c210 tx=0x7f70e001f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c4c6b36891ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0007d40 con 0x7f70d0006a20 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e0005b00 con 0x7f70d0006a20 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70dbfff640 1 -- 172.21.15.106:0/2374509018 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0007070 con 0x7f70d0006a20 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70eb6e3640 1 -- 172.21.15.106:0/2374509018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d0006a20 msgr2=0x7f70d0006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70eb6e3640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d0006a20 0x7f70d0006e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f70e000c210 tx=0x7f70e001f2f0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70eb6e3640 1 -- 172.21.15.106:0/2374509018 shutdown_connections 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70eb6e3640 1 --2- 172.21.15.106:0/2374509018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d0006a20 0x7f70d0006e30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70eb6e3640 1 -- 172.21.15.106:0/2374509018 >> 172.21.15.106:0/2374509018 conn(0x7f70e409ba30 msgr2=0x7f70e40b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70eb6e3640 1 -- 172.21.15.106:0/2374509018 shutdown_connections 2024-04-25T11:20:23.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.543+0000 7f70eb6e3640 1 -- 172.21.15.106:0/2374509018 wait complete. 2024-04-25T11:20:23.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 Processor -- start 2024-04-25T11:20:23.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 -- start start 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d0006a20 0x7f70e413b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e413eac0 0x7f70e415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70e415b480 0x7f70e4148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40b12d0 con 0x7f70e415b480 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40b1150 con 0x7f70e413eac0 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f70e40b6840 con 0x7f70d0006a20 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70e415b480 0x7f70e4148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70e415b480 0x7f70e4148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59412/0 (socket says 172.21.15.106:59412) 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 -- 172.21.15.106:0/621145493 learned_addr learned my addr 172.21.15.106:0/621145493 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e947f640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d0006a20 0x7f70e413b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e8c7e640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e413eac0 0x7f70e415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 -- 172.21.15.106:0/621145493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d0006a20 msgr2=0x7f70e413b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d0006a20 0x7f70e413b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 -- 172.21.15.106:0/621145493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e413eac0 msgr2=0x7f70e415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e413eac0 0x7f70e415af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 -- 172.21.15.106:0/621145493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e415b890 con 0x7f70e415b480 2024-04-25T11:20:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70e9c80640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70e415b480 0x7f70e4148bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f70e40ce670 tx=0x7f70d400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70d9ffb640 1 -- 172.21.15.106:0/621145493 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4016620 con 0x7f70e415b480 2024-04-25T11:20:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70d9ffb640 1 -- 172.21.15.106:0/621145493 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d401b070 con 0x7f70e415b480 2024-04-25T11:20:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e413bd50 con 0x7f70e415b480 2024-04-25T11:20:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70d9ffb640 1 -- 172.21.15.106:0/621145493 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d40169e0 con 0x7f70e415b480 2024-04-25T11:20:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.547+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e414e7a0 con 0x7f70e415b480 2024-04-25T11:20:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.551+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e414e210 con 0x7f70e415b480 2024-04-25T11:20:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.551+0000 7f70d9ffb640 1 -- 172.21.15.106:0/621145493 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f70d4016b80 con 0x7f70e415b480 2024-04-25T11:20:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.551+0000 7f70d9ffb640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f70c806f400 0x7f70c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.551+0000 7f70d9ffb640 1 -- 172.21.15.106:0/621145493 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f70d4090070 con 0x7f70e415b480 2024-04-25T11:20:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.551+0000 7f70e947f640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f70c806f400 0x7f70c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.551+0000 7f70e947f640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f70c806f400 0x7f70c8071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70dc000c10 tx=0x7f70dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.555+0000 7f70d9ffb640 1 -- 172.21.15.106:0/621145493 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f70e415b890 con 0x7f70e415b480 2024-04-25T11:20:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:23.723+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f70e40a0780 con 0x7f70e415b480 2024-04-25T11:20:24.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.351+0000 7f70d9ffb640 1 -- 172.21.15.106:0/621145493 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f70d405d040 con 0x7f70e415b480 2024-04-25T11:20:24.351 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-25T11:20:24.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f70c806f400 msgr2=0x7f70c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f70c806f400 0x7f70c8071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70dc000c10 tx=0x7f70dc009040 comp rx=0 tx=0).stop 2024-04-25T11:20:24.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70e415b480 msgr2=0x7f70e4148bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70e415b480 0x7f70e4148bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f70e40ce670 tx=0x7f70d400f520 comp rx=0 tx=0).stop 2024-04-25T11:20:24.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 shutdown_connections 2024-04-25T11:20:24.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f70c806f400 0x7f70c8071910 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70dc000c10 tx=0x7f70dc009040 comp rx=0 tx=0).stop 2024-04-25T11:20:24.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70e415b480 0x7f70e4148bd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e413eac0 0x7f70e415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.355+0000 7f70eb6e3640 1 --2- 172.21.15.106:0/621145493 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70d0006a20 0x7f70e413b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.359+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 >> 172.21.15.106:0/621145493 conn(0x7f70e409ba30 msgr2=0x7f70e4070130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:24.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.359+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 shutdown_connections 2024-04-25T11:20:24.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.359+0000 7f70eb6e3640 1 -- 172.21.15.106:0/621145493 wait complete. 2024-04-25T11:20:24.384 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:20:24.384 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:20:24.385 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:20:24.385 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:24.385 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:20:24.385 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:20:24.385 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:20:24.385 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:24.386 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:20:24.386 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:20:24.386 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:20:24.386 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:24.386 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:20:24.386 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:20:24.386 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:20:24.387 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:24.403 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:20:24.403 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:20:24.404 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:20:24.404 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:20:24.405 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:20:24.406 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:20:24.406 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:20:24.407 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:20:24.447 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:20:24.447 DEBUG:teuthology.parallel:result is None 2024-04-25T11:20:24.448 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:20:24.448 DEBUG:teuthology.parallel:result is None 2024-04-25T11:20:24.449 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:20:24.449 DEBUG:teuthology.parallel:result is None 2024-04-25T11:20:24.450 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:20:24.450 DEBUG:teuthology.parallel:result is None 2024-04-25T11:20:24.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:20:24.542 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-25T11:20:24.545 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-25T11:20:24.550 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-04-25T11:20:24.552 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-04-25T11:20:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.599+0000 7f9caabc7640 1 Processor -- start 2024-04-25T11:20:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.599+0000 7f9caabc7640 1 -- start start 2024-04-25T11:20:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9caabc7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ca40cf0e0 0x7f9ca409bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9caabc7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ca409c140 0x7f9ca40d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9caabc7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca40d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9caabc7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca409ce70 con 0x7f9ca409c140 2024-04-25T11:20:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9caabc7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca40b8ce0 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9caabc7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca40b46f0 con 0x7f9ca40d1680 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ca409c140 0x7f9ca40d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ca409c140 0x7f9ca40d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59452/0 (socket says 172.21.15.106:59452) 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 -- 172.21.15.106:0/753334639 learned_addr learned my addr 172.21.15.106:0/753334639 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca9164640 1 --2- 172.21.15.106:0/753334639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca40d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca8963640 1 --2- 172.21.15.106:0/753334639 >> v2:172.21.15.134:3300/0 conn(0x7f9ca40cf0e0 0x7f9ca409bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 -- 172.21.15.106:0/753334639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 msgr2=0x7f9ca40d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 --2- 172.21.15.106:0/753334639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca40d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 -- 172.21.15.106:0/753334639 >> v2:172.21.15.134:3300/0 conn(0x7f9ca40cf0e0 msgr2=0x7f9ca409bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 --2- 172.21.15.106:0/753334639 >> v2:172.21.15.134:3300/0 conn(0x7f9ca40cf0e0 0x7f9ca409bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 -- 172.21.15.106:0/753334639 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca40cf4f0 con 0x7f9ca409c140 2024-04-25T11:20:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca8963640 1 --2- 172.21.15.106:0/753334639 >> v2:172.21.15.134:3300/0 conn(0x7f9ca40cf0e0 0x7f9ca409bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.603+0000 7f9ca3fff640 1 --2- 172.21.15.106:0/753334639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca409c140 0x7f9ca40d1140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40747c0 tx=0x7f9c9801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460467137e06148e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9ca37fe640 1 -- 172.21.15.106:0/753334639 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980a6040 con 0x7f9ca409c140 2024-04-25T11:20:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9ca37fe640 1 -- 172.21.15.106:0/753334639 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c980af070 con 0x7f9ca409c140 2024-04-25T11:20:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9ca37fe640 1 -- 172.21.15.106:0/753334639 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980aa860 con 0x7f9ca409c140 2024-04-25T11:20:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- 172.21.15.106:0/753334639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca409c140 msgr2=0x7f9ca40d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 --2- 172.21.15.106:0/753334639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca409c140 0x7f9ca40d1140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40747c0 tx=0x7f9c9801c520 comp rx=0 tx=0).stop 2024-04-25T11:20:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- 172.21.15.106:0/753334639 shutdown_connections 2024-04-25T11:20:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 --2- 172.21.15.106:0/753334639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca40d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 --2- 172.21.15.106:0/753334639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca409c140 0x7f9ca40d1140 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 --2- 172.21.15.106:0/753334639 >> v2:172.21.15.134:3300/0 conn(0x7f9ca40cf0e0 0x7f9ca409bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- 172.21.15.106:0/753334639 >> 172.21.15.106:0/753334639 conn(0x7f9ca40b5060 msgr2=0x7f9ca40b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- 172.21.15.106:0/753334639 shutdown_connections 2024-04-25T11:20:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- 172.21.15.106:0/753334639 wait complete. 2024-04-25T11:20:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 Processor -- start 2024-04-25T11:20:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- start start 2024-04-25T11:20:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca409c140 0x7f9ca40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca40cf0e0 0x7f9ca41670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca41675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca409c550 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca409c870 con 0x7f9ca409c140 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.607+0000 7f9caabc7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca40b8e80 con 0x7f9ca40d1680 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca40cf0e0 0x7f9ca41670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca40cf0e0 0x7f9ca41670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59462/0 (socket says 172.21.15.106:59462) 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 -- 172.21.15.106:0/238506839 learned_addr learned my addr 172.21.15.106:0/238506839 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca8963640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca409c140 0x7f9ca40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca9164640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca41675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 -- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 msgr2=0x7f9ca41675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca41675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 -- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca409c140 msgr2=0x7f9ca40c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca409c140 0x7f9ca40c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 -- 172.21.15.106:0/238506839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca40ca670 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca9164640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca41675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca8963640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca409c140 0x7f9ca40c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca3fff640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca40cf0e0 0x7f9ca41670b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9c9801abd0 tx=0x7f9c9801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca17fa640 1 -- 172.21.15.106:0/238506839 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980bd5f0 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca17fa640 1 -- 172.21.15.106:0/238506839 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c980a6030 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca40ca950 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca17fa640 1 -- 172.21.15.106:0/238506839 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980bd990 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca40cc4b0 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca4050750 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca17fa640 1 -- 172.21.15.106:0/238506839 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9c980c63f0 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca17fa640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9c9006f430 0x7f9c90071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca17fa640 1 -- 172.21.15.106:0/238506839 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f9c98129470 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.611+0000 7f9ca8963640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9c9006f430 0x7f9c90071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.615+0000 7f9ca8963640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9c9006f430 0x7f9c90071940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c003fd0 tx=0x7f9c9c0039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.615+0000 7f9ca17fa640 1 -- 172.21.15.106:0/238506839 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c98016110 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:24.787+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9ca4072b40 con 0x7f9ca40cf0e0 2024-04-25T11:20:24.985 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:20:24.988 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:20:25.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.371+0000 7f9ca17fa640 1 -- 172.21.15.106:0/238506839 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f9c980f6860 con 0x7f9ca40cf0e0 2024-04-25T11:20:25.372 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-25T11:20:25.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9c9006f430 msgr2=0x7f9c90071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9c9006f430 0x7f9c90071940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c003fd0 tx=0x7f9c9c0039a0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca40cf0e0 msgr2=0x7f9ca41670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca40cf0e0 0x7f9ca41670b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9c9801abd0 tx=0x7f9c9801ad60 comp rx=0 tx=0).stop 2024-04-25T11:20:25.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 shutdown_connections 2024-04-25T11:20:25.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca40d1680 0x7f9ca41675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9c9006f430 0x7f9c90071940 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca40cf0e0 0x7f9ca41670b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 --2- 172.21.15.106:0/238506839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca409c140 0x7f9ca40c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 >> 172.21.15.106:0/238506839 conn(0x7f9ca40b5060 msgr2=0x7f9ca4158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:25.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 shutdown_connections 2024-04-25T11:20:25.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.375+0000 7f9caabc7640 1 -- 172.21.15.106:0/238506839 wait complete. 2024-04-25T11:20:25.401 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:20:25.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd86d9f640 1 Processor -- start 2024-04-25T11:20:25.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd86d9f640 1 -- start start 2024-04-25T11:20:25.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd86d9f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd80160f10 0x7fbd8015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd86d9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8015d6f0 0x7fbd8015db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd86d9f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbd800b43f0 0x7fbd800b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd86d9f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbd800b5cc0 con 0x7fbd80160f10 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd86d9f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8015cf50 con 0x7fbd800b43f0 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd86d9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800b5f80 con 0x7fbd8015d6f0 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd84b3b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd80160f10 0x7fbd8015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd84b3b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd80160f10 0x7fbd8015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57252/0 (socket says 172.21.15.106:57252) 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd84b3b640 1 -- 172.21.15.106:0/2795974482 learned_addr learned my addr 172.21.15.106:0/2795974482 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd8533c640 1 --2- 172.21.15.106:0/2795974482 >> v2:172.21.15.134:3300/0 conn(0x7fbd800b43f0 0x7fbd800b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd7ffff640 1 --2- 172.21.15.106:0/2795974482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8015d6f0 0x7fbd8015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd84b3b640 1 -- 172.21.15.106:0/2795974482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8015d6f0 msgr2=0x7fbd8015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd84b3b640 1 --2- 172.21.15.106:0/2795974482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8015d6f0 0x7fbd8015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd84b3b640 1 -- 172.21.15.106:0/2795974482 >> v2:172.21.15.134:3300/0 conn(0x7fbd800b43f0 msgr2=0x7fbd800b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd84b3b640 1 --2- 172.21.15.106:0/2795974482 >> v2:172.21.15.134:3300/0 conn(0x7fbd800b43f0 0x7fbd800b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.551+0000 7fbd84b3b640 1 -- 172.21.15.106:0/2795974482 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd800b6260 con 0x7fbd80160f10 2024-04-25T11:20:25.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd84b3b640 1 --2- 172.21.15.106:0/2795974482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd80160f10 0x7fbd8015ca10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbd80073c80 tx=0x7fbd7001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc0731080872cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd7f7fe640 1 -- 172.21.15.106:0/2795974482 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700a57b0 con 0x7fbd80160f10 2024-04-25T11:20:25.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd7f7fe640 1 -- 172.21.15.106:0/2795974482 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd700a5950 con 0x7fbd80160f10 2024-04-25T11:20:25.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd7f7fe640 1 -- 172.21.15.106:0/2795974482 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700a5d20 con 0x7fbd80160f10 2024-04-25T11:20:25.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd86d9f640 1 -- 172.21.15.106:0/2795974482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd80160f10 msgr2=0x7fbd8015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/2795974482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd80160f10 0x7fbd8015ca10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbd80073c80 tx=0x7fbd7001c930 comp rx=0 tx=0).stop 2024-04-25T11:20:25.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd86d9f640 1 -- 172.21.15.106:0/2795974482 shutdown_connections 2024-04-25T11:20:25.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/2795974482 >> v2:172.21.15.134:3300/0 conn(0x7fbd800b43f0 0x7fbd800b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/2795974482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd8015d6f0 0x7fbd8015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/2795974482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd80160f10 0x7fbd8015ca10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.555+0000 7fbd86d9f640 1 -- 172.21.15.106:0/2795974482 >> 172.21.15.106:0/2795974482 conn(0x7fbd8006c7c0 msgr2=0x7fbd8006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:25.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 -- 172.21.15.106:0/2795974482 shutdown_connections 2024-04-25T11:20:25.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 -- 172.21.15.106:0/2795974482 wait complete. 2024-04-25T11:20:25.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 Processor -- start 2024-04-25T11:20:25.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 -- start start 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd800b43f0 0x7fbd800b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8015d6f0 0x7fbd80151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd80160f10 0x7fbd800b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800d19b0 con 0x7fbd8015d6f0 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800b4d40 con 0x7fbd800b43f0 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd86d9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800d1b30 con 0x7fbd80160f10 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd7ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8015d6f0 0x7fbd80151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd7ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8015d6f0 0x7fbd80151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57254/0 (socket says 172.21.15.106:57254) 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd7ffff640 1 -- 172.21.15.106:0/4266236757 learned_addr learned my addr 172.21.15.106:0/4266236757 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd8533c640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd80160f10 0x7fbd800b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd84b3b640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd800b43f0 0x7fbd800b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd7ffff640 1 -- 172.21.15.106:0/4266236757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd80160f10 msgr2=0x7fbd800b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd7ffff640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd80160f10 0x7fbd800b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd7ffff640 1 -- 172.21.15.106:0/4266236757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd800b43f0 msgr2=0x7fbd800b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd7ffff640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd800b43f0 0x7fbd800b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.559+0000 7fbd7ffff640 1 -- 172.21.15.106:0/4266236757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8015e6c0 con 0x7fbd8015d6f0 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd7ffff640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8015d6f0 0x7fbd80151e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800d5f0 tx=0x7fbd6800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd7d7fa640 1 -- 172.21.15.106:0/4266236757 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680125f0 con 0x7fbd8015d6f0 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd7d7fa640 1 -- 172.21.15.106:0/4266236757 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd68019070 con 0x7fbd8015d6f0 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd7d7fa640 1 -- 172.21.15.106:0/4266236757 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680129b0 con 0x7fbd8015d6f0 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8015e9a0 con 0x7fbd8015d6f0 2024-04-25T11:20:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd80154910 con 0x7fbd8015d6f0 2024-04-25T11:20:25.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8015eba0 con 0x7fbd8015d6f0 2024-04-25T11:20:25.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd7d7fa640 1 -- 172.21.15.106:0/4266236757 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbd68012b50 con 0x7fbd8015d6f0 2024-04-25T11:20:25.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd7d7fa640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbd6006f400 0x7fbd60071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd84b3b640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbd6006f400 0x7fbd60071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd7d7fa640 1 -- 172.21.15.106:0/4266236757 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbd6808f350 con 0x7fbd8015d6f0 2024-04-25T11:20:25.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.563+0000 7fbd84b3b640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbd6006f400 0x7fbd60071910 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd700ae000 tx=0x7fbd700a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.567+0000 7fbd7d7fa640 1 -- 172.21.15.106:0/4266236757 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbd8015eba0 con 0x7fbd8015d6f0 2024-04-25T11:20:25.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbd800b3950 con 0x7fbd8015d6f0 2024-04-25T11:20:25.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd7d7fa640 1 -- 172.21.15.106:0/4266236757 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fbd6805c590 con 0x7fbd8015d6f0 2024-04-25T11:20:25.778 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:25.778 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQACPCpmt6nyOBAAujM4DC6/QuUUc3m9T3BnWg==", 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-25T11:20:25.779 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQADPCpmzp9PChAA772mlqVvY6+7zwkaj1ImZg==", 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQADPCpmd/nIBBAAhxEcRyNRhKxwIP7z11osNg==", 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-25T11:20:25.780 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQADPCpm64lQEhAAVTJ1nmvnVVqHefhc5uonnw==", 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.781 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAGPCpm58GZHxAA4ukc6lqPm3tuB+nVCWvflg==", 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAJPCpmYocQJxAA4kmFs222wtG4o6KtWq8SHA==", 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:25.782 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAMPCpmjaCMLRAAhEZEfHFGfbp6h8I5Whkidw==", 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.783 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAPPCpmrpT0NBAATZTX0OHG9n4x5UpHopwzzg==", 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAVPCpmU2j/NBAAJ18g1a7XKUDnxp1K5GZYIQ==", 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.784 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAYPCpmXGJYOxAAMaE55oFohnPXLoNNOiSiGw==", 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAcPCpmBVEdBhAAh/QAN9kWd9QpIivo1WQgKw==", 2024-04-25T11:20:25.785 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAfPCpmCg5ZDRAApPy0RHPegmvJr1R7Vvu/ew==", 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.786 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQADPCpmT+VXFxAASImB5WSW6aNZRALgQGZkFQ==", 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQADPCpm83KiIBAAdXUTXe8EL2RdKF4EsqCYiQ==", 2024-04-25T11:20:25.787 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQADPCpm5uLmGxAAml2ZRg40PRw7bkxpM6f2gA==", 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-25T11:20:25.788 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQADPCpmSX8cJRAAKVq+BB+WPau/0qh0oK7z8g==", 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-25T11:20:25.789 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQACPCpmXcv0GxAAPcM8adbHSeaj/Br9OJ3wTg==", 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-mds", 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAlPCpmPjm1GxAAGNR83H46xa356nastE8Iqg==", 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAlPCpm/la1GxAA7/5zJ1TN5mf7fxLB7e2/LQ==", 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-osd", 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAlPCpmsW+1GxAAnJPdiImM2JAJ6nXS6Ejf7Q==", 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.791 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAlPCpmHo21GxAA0RIkqajAIy23CFRkfq/n1A==", 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAlPCpmtKa1GxAAzgHFvcIYvdQOjXnEaJ/bYw==", 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAlPCpm48C1GxAAUsnr1879wL8Pbjih+pMCOw==", 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQACPCpmAIPHMxAACFuC5wW8a+Lhyl4gy+aOuQ==", 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-25T11:20:25.793 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQACPCpmEcHrLhAAoHf/nv5AgPZKvsAjrH/bsQ==", 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T11:20:25.794 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:20:25.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbd6006f400 msgr2=0x7fbd60071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbd6006f400 0x7fbd60071910 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd700ae000 tx=0x7fbd700a7070 comp rx=0 tx=0).stop 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8015d6f0 msgr2=0x7fbd80151e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8015d6f0 0x7fbd80151e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800d5f0 tx=0x7fbd6800dab0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 shutdown_connections 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd80160f10 0x7fbd800b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fbd6006f400 0x7fbd60071910 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8015d6f0 0x7fbd80151e70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 --2- 172.21.15.106:0/4266236757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd800b43f0 0x7fbd800b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 >> 172.21.15.106:0/4266236757 conn(0x7fbd8006c7c0 msgr2=0x7fbd80155d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 shutdown_connections 2024-04-25T11:20:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.779+0000 7fbd86d9f640 1 -- 172.21.15.106:0/4266236757 wait complete. 2024-04-25T11:20:25.799 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T11:20:25.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5ea6f640 1 Processor -- start 2024-04-25T11:20:25.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5ea6f640 1 -- start start 2024-04-25T11:20:25.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5ea6f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7e580d1e20 0x7f7e580cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5ea6f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e581604c0 con 0x7f7e580b2ca0 2024-04-25T11:20:25.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5ea6f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e580b3980 con 0x7f7e580ccac0 2024-04-25T11:20:25.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5ea6f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7e580b30b0 con 0x7f7e580d1e20 2024-04-25T11:20:25.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5c80b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7e580b2ca0 0x7f7e5806cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34416/0 (socket says 172.21.15.106:34416) 2024-04-25T11:20:25.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5c80b640 1 -- 172.21.15.106:0/1241349172 learned_addr learned my addr 172.21.15.106:0/1241349172 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:25.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e5d00c640 1 --2- 172.21.15.106:0/1241349172 >> v2:172.21.15.134:3300/0 conn(0x7f7e580d1e20 0x7f7e580cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470565704 0 0) 0x7f7e581604c0 con 0x7f7e580b2ca0 2024-04-25T11:20:25.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e38003530 con 0x7f7e580b2ca0 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.935+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1735864080 0 0) 0x7f7e38003530 con 0x7f7e580b2ca0 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 >> v2:172.21.15.134:3300/0 conn(0x7f7e580d1e20 msgr2=0x7f7e580cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 --2- 172.21.15.106:0/1241349172 >> v2:172.21.15.134:3300/0 conn(0x7f7e580d1e20 0x7f7e580cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 >> v1:172.21.15.134:6789/0 conn(0x7f7e580ccac0 legacy=0x7f7e580d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e580b3c60 con 0x7f7e580b2ca0 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f7e48001680 con 0x7f7e580b2ca0 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 >> v1:172.21.15.106:6789/0 conn(0x7f7e580b2ca0 legacy=0x7f7e5806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e380039c0 0x7f7e38003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e38004810 0x7f7e38004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380051d0 0x7f7e3800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3800c140 con 0x7f7e380039c0 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3800c2f0 con 0x7f7e38004810 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3800c4f0 con 0x7f7e380051d0 2024-04-25T11:20:25.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5c80b640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e380039c0 0x7f7e38003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5c80b640 1 -- 172.21.15.106:0/1241349172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380051d0 msgr2=0x7f7e3800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:25.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5c80b640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380051d0 0x7f7e3800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5c80b640 1 -- 172.21.15.106:0/1241349172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e38004810 msgr2=0x7f7e38004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:25.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5c80b640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e38004810 0x7f7e38004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5d00c640 1 -- 172.21.15.106:0/1241349172 reap_dead start 2024-04-25T11:20:25.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5c80b640 1 -- 172.21.15.106:0/1241349172 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e38003530 con 0x7f7e380039c0 2024-04-25T11:20:25.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5c80b640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e380039c0 0x7f7e38003e30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e58163b10 tx=0x7f7e4801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daad3be05bbaa131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48007d40 con 0x7f7e380039c0 2024-04-25T11:20:25.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e480a9070 con 0x7f7e380039c0 2024-04-25T11:20:25.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e577fe640 1 -- 172.21.15.106:0/1241349172 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48005d80 con 0x7f7e380039c0 2024-04-25T11:20:25.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/1241349172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e380039c0 msgr2=0x7f7e38003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5ea6f640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e380039c0 0x7f7e38003e30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e58163b10 tx=0x7f7e4801cb60 comp rx=0 tx=0).stop 2024-04-25T11:20:25.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/1241349172 shutdown_connections 2024-04-25T11:20:25.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5ea6f640 1 --2- 172.21.15.106:0/1241349172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e380039c0 0x7f7e38003e30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/1241349172 >> 172.21.15.106:0/1241349172 conn(0x7f7e580b4020 msgr2=0x7f7e580b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:25.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/1241349172 shutdown_connections 2024-04-25T11:20:25.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.939+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/1241349172 wait complete. 2024-04-25T11:20:25.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.943+0000 7f7e5ea6f640 1 Processor -- start 2024-04-25T11:20:25.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.943+0000 7f7e5ea6f640 1 -- start start 2024-04-25T11:20:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380039c0 0x7f7e5809dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5809f6d0 0x7f7e5809fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e580b0db0 0x7f7e580b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5806d3e0 con 0x7f7e580b0db0 2024-04-25T11:20:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e580ce750 con 0x7f7e5809f6d0 2024-04-25T11:20:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5806d560 con 0x7f7e380039c0 2024-04-25T11:20:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e580b0db0 0x7f7e580b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e580b0db0 0x7f7e580b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57280/0 (socket says 172.21.15.106:57280) 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 -- 172.21.15.106:0/3338020839 learned_addr learned my addr 172.21.15.106:0/3338020839 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e57fff640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5809f6d0 0x7f7e5809fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5c80b640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380039c0 0x7f7e5809dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 -- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380039c0 msgr2=0x7f7e5809dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380039c0 0x7f7e5809dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 -- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5809f6d0 msgr2=0x7f7e5809fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5809f6d0 0x7f7e5809fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 -- 172.21.15.106:0/3338020839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5809e270 con 0x7f7e580b0db0 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5c80b640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380039c0 0x7f7e5809dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5d00c640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e580b0db0 0x7f7e580b11c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e580cca50 tx=0x7f7e5000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e557fa640 1 -- 172.21.15.106:0/3338020839 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50018020 con 0x7f7e580b0db0 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e557fa640 1 -- 172.21.15.106:0/3338020839 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5000ba80 con 0x7f7e580b0db0 2024-04-25T11:20:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e581fa900 con 0x7f7e580b0db0 2024-04-25T11:20:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e557fa640 1 -- 172.21.15.106:0/3338020839 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000be50 con 0x7f7e580b0db0 2024-04-25T11:20:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e581f9b40 con 0x7f7e580b0db0 2024-04-25T11:20:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e580746e0 con 0x7f7e580b0db0 2024-04-25T11:20:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e557fa640 1 -- 172.21.15.106:0/3338020839 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7e5001b020 con 0x7f7e580b0db0 2024-04-25T11:20:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e557fa640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7e3c06f5a0 0x7f7e3c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e557fa640 1 -- 172.21.15.106:0/3338020839 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f7e5008f4b0 con 0x7f7e580b0db0 2024-04-25T11:20:25.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5c80b640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7e3c06f5a0 0x7f7e3c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.947+0000 7f7e5c80b640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7e3c06f5a0 0x7f7e3c071ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7e480ab000 tx=0x7f7e480a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:25.951+0000 7f7e557fa640 1 -- 172.21.15.106:0/3338020839 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e50022020 con 0x7f7e580b0db0 2024-04-25T11:20:26.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.155+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f7e580a0450 con 0x7f7e580b0db0 2024-04-25T11:20:26.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.159+0000 7f7e557fa640 1 -- 172.21.15.106:0/3338020839 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f7e5809e270 con 0x7f7e580b0db0 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7e3c06f5a0 msgr2=0x7f7e3c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7e3c06f5a0 0x7f7e3c071ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7e480ab000 tx=0x7f7e480a7070 comp rx=0 tx=0).stop 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e580b0db0 msgr2=0x7f7e580b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e580b0db0 0x7f7e580b11c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e580cca50 tx=0x7f7e5000b480 comp rx=0 tx=0).stop 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 shutdown_connections 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7e3c06f5a0 0x7f7e3c071ab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e580b0db0 0x7f7e580b11c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5809f6d0 0x7f7e5809fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 --2- 172.21.15.106:0/3338020839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e380039c0 0x7f7e5809dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 >> 172.21.15.106:0/3338020839 conn(0x7f7e580b4020 msgr2=0x7f7e5806fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 shutdown_connections 2024-04-25T11:20:26.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.163+0000 7f7e5ea6f640 1 -- 172.21.15.106:0/3338020839 wait complete. 2024-04-25T11:20:26.181 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T11:20:26.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8816671640 1 Processor -- start 2024-04-25T11:20:26.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8816671640 1 -- start start 2024-04-25T11:20:26.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8816671640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f88100af4a0 0x7f88100d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8816671640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8816671640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88100b5150 con 0x7f881006cb30 2024-04-25T11:20:26.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8816671640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f88100b4940 con 0x7f88100af4a0 2024-04-25T11:20:26.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8816671640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f881006cf40 con 0x7f881015ebe0 2024-04-25T11:20:26.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f880f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f880ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f88100af4a0 0x7f88100d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8814c0e640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f881006cb30 0x7f881015f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:35162/0 (socket says 172.21.15.106:35162) 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f880f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:35982/0 (socket says 172.21.15.106:35982) 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f880ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f88100af4a0 0x7f88100d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:45068/0 (socket says 172.21.15.106:45068) 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.323+0000 7f8814c0e640 1 -- 172.21.15.106:0/3735743310 learned_addr learned my addr 172.21.15.106:0/3735743310 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f880ffff640 1 -- 172.21.15.106:0/3735743310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 msgr2=0x7f881006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f880ffff640 1 --2- 172.21.15.106:0/3735743310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f880ffff640 1 -- 172.21.15.106:0/3735743310 >> v1:172.21.15.134:6789/0 conn(0x7f881006cb30 legacy=0x7f881015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f880ffff640 1 -- 172.21.15.106:0/3735743310 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88100af9d0 con 0x7f88100af4a0 2024-04-25T11:20:26.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f880ffff640 1 --2- 172.21.15.106:0/3735743310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100af4a0 0x7f88100d1e80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8808009d90 tx=0x7f880801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=133949ac5f42e6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:26.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f880effd640 1 -- 172.21.15.106:0/3735743310 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080b2070 con 0x7f88100af4a0 2024-04-25T11:20:26.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f880effd640 1 -- 172.21.15.106:0/3735743310 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88080a5940 con 0x7f88100af4a0 2024-04-25T11:20:26.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f880effd640 1 -- 172.21.15.106:0/3735743310 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080a5d10 con 0x7f88100af4a0 2024-04-25T11:20:26.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f8816671640 1 -- 172.21.15.106:0/3735743310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100af4a0 msgr2=0x7f88100d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f8816671640 1 --2- 172.21.15.106:0/3735743310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100af4a0 0x7f88100d1e80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8808009d90 tx=0x7f880801c970 comp rx=0 tx=0).stop 2024-04-25T11:20:26.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f8816671640 1 -- 172.21.15.106:0/3735743310 shutdown_connections 2024-04-25T11:20:26.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f8816671640 1 --2- 172.21.15.106:0/3735743310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f8816671640 1 --2- 172.21.15.106:0/3735743310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100af4a0 0x7f88100d1e80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f8816671640 1 -- 172.21.15.106:0/3735743310 >> 172.21.15.106:0/3735743310 conn(0x7f881009ba30 msgr2=0x7f88100b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:26.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f8816671640 1 -- 172.21.15.106:0/3735743310 shutdown_connections 2024-04-25T11:20:26.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.327+0000 7f8816671640 1 -- 172.21.15.106:0/3735743310 wait complete. 2024-04-25T11:20:26.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 Processor -- start 2024-04-25T11:20:26.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 -- start start 2024-04-25T11:20:26.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f881006cb30 0x7f88101fb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88100af4a0 0x7f881015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881014c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f881015ffc0 con 0x7f88100af4a0 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f881015fe40 con 0x7f881006cb30 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f881016b300 con 0x7f881015ebe0 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88100af4a0 0x7f881015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88100af4a0 0x7f881015aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57294/0 (socket says 172.21.15.106:57294) 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 -- 172.21.15.106:0/3939422757 learned_addr learned my addr 172.21.15.106:0/3939422757 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880ffff640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f881006cb30 0x7f88101fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 -- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 msgr2=0x7f881014c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8814c0e640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881014c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881014c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 -- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f881006cb30 msgr2=0x7f88101fb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f881006cb30 0x7f88101fb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 -- 172.21.15.106:0/3939422757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881015afb0 con 0x7f88100af4a0 2024-04-25T11:20:26.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880ffff640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f881006cb30 0x7f88101fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:26.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880f7fe640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88100af4a0 0x7f881015aa70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f87f800e8a0 tx=0x7f87f800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:26.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880cff9640 1 -- 172.21.15.106:0/3939422757 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8017070 con 0x7f88100af4a0 2024-04-25T11:20:26.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880cff9640 1 -- 172.21.15.106:0/3939422757 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f800c940 con 0x7f88100af4a0 2024-04-25T11:20:26.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88100b4230 con 0x7f88100af4a0 2024-04-25T11:20:26.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f880cff9640 1 -- 172.21.15.106:0/3939422757 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800ccf0 con 0x7f88100af4a0 2024-04-25T11:20:26.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.331+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88101faf40 con 0x7f88100af4a0 2024-04-25T11:20:26.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.335+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88100b3fa0 con 0x7f88100af4a0 2024-04-25T11:20:26.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.335+0000 7f880cff9640 1 -- 172.21.15.106:0/3939422757 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f87f801a070 con 0x7f88100af4a0 2024-04-25T11:20:26.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.335+0000 7f880cff9640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f87f006f400 0x7f87f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.335+0000 7f880cff9640 1 -- 172.21.15.106:0/3939422757 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f87f808f6c0 con 0x7f88100af4a0 2024-04-25T11:20:26.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.335+0000 7f880ffff640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f87f006f400 0x7f87f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.335+0000 7f880ffff640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f87f006f400 0x7f87f0071910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f88080af000 tx=0x7f88080a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:26.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.339+0000 7f880cff9640 1 -- 172.21.15.106:0/3939422757 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87f805c900 con 0x7f88100af4a0 2024-04-25T11:20:26.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.563+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f8810050750 con 0x7f88100af4a0 2024-04-25T11:20:26.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.567+0000 7f880cff9640 1 -- 172.21.15.106:0/3939422757 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f87f805c200 con 0x7f88100af4a0 2024-04-25T11:20:26.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f87f006f400 msgr2=0x7f87f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f87f006f400 0x7f87f0071910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f88080af000 tx=0x7f88080a7070 comp rx=0 tx=0).stop 2024-04-25T11:20:26.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88100af4a0 msgr2=0x7f881015aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88100af4a0 0x7f881015aa70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f87f800e8a0 tx=0x7f87f800eca0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 shutdown_connections 2024-04-25T11:20:26.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881015ebe0 0x7f881014c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f87f006f400 0x7f87f0071910 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88100af4a0 0x7f881015aa70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 --2- 172.21.15.106:0/3939422757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f881006cb30 0x7f88101fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 >> 172.21.15.106:0/3939422757 conn(0x7f881009ba30 msgr2=0x7f88100704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:26.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 shutdown_connections 2024-04-25T11:20:26.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.571+0000 7f8816671640 1 -- 172.21.15.106:0/3939422757 wait complete. 2024-04-25T11:20:26.588 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T11:20:26.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.723+0000 7fa0a8089640 1 Processor -- start 2024-04-25T11:20:26.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.723+0000 7fa0a8089640 1 -- start start 2024-04-25T11:20:26.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a8089640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a00c6c70 0x7fa0a00d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a8089640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa0a00d1f60 0x7fa0a00b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a8089640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a00d2370 con 0x7fa0a009cbf0 2024-04-25T11:20:26.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a8089640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a00cf4f0 con 0x7fa0a00d1f60 2024-04-25T11:20:26.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a8089640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a00b8bb0 con 0x7fa0a00c6c70 2024-04-25T11:20:26.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a5624640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa0a009cbf0 0x7fa0a00b6d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34432/0 (socket says 172.21.15.106:34432) 2024-04-25T11:20:26.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a5624640 1 -- 172.21.15.106:0/969524407 learned_addr learned my addr 172.21.15.106:0/969524407 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:26.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a6626640 1 --2- 172.21.15.106:0/969524407 >> v2:172.21.15.134:3300/0 conn(0x7fa0a00d1f60 0x7fa0a00b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149177661 0 0) 0x7fa0a00d2370 con 0x7fa0a009cbf0 2024-04-25T11:20:26.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa08c003530 con 0x7fa0a009cbf0 2024-04-25T11:20:26.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a5e25640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a00c6c70 0x7fa0a00d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3196312988 0 0) 0x7fa08c003530 con 0x7fa0a009cbf0 2024-04-25T11:20:26.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a00c6c70 msgr2=0x7fa0a00d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:26.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a00c6c70 0x7fa0a00d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 >> v2:172.21.15.134:3300/0 conn(0x7fa0a00d1f60 msgr2=0x7fa0a00b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:26.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 --2- 172.21.15.106:0/969524407 >> v2:172.21.15.134:3300/0 conn(0x7fa0a00d1f60 0x7fa0a00b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a00b79d0 con 0x7fa0a009cbf0 2024-04-25T11:20:26.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fa090001680 con 0x7fa0a009cbf0 2024-04-25T11:20:26.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 >> v1:172.21.15.106:6789/0 conn(0x7fa0a009cbf0 legacy=0x7fa0a00b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c0039c0 0x7fa08c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a00c6c70 0x7fa08c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa08c004d50 0x7fa08c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a00cf4f0 con 0x7fa08c004d50 2024-04-25T11:20:26.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a00b8bb0 con 0x7fa08c0039c0 2024-04-25T11:20:26.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c00ba50 con 0x7fa0a00c6c70 2024-04-25T11:20:26.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a6626640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa08c004d50 0x7fa08c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a5624640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a00c6c70 0x7fa08c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a6626640 1 -- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a00c6c70 msgr2=0x7fa08c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a6626640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a00c6c70 0x7fa08c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a5e25640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c0039c0 0x7fa08c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a6626640 1 -- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c0039c0 msgr2=0x7fa08c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a6626640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c0039c0 0x7fa08c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a6626640 1 -- 172.21.15.106:0/969524407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c003530 con 0x7fa08c004d50 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.727+0000 7fa0a5e25640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c0039c0 0x7fa08c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a6626640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa08c004d50 0x7fa08c005160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa098009fc0 tx=0x7fa09801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01f68379b6da7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980b2070 con 0x7fa08c004d50 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa090001d40 con 0x7fa08c004d50 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a4e23640 1 -- 172.21.15.106:0/969524407 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098009c10 con 0x7fa08c004d50 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a8089640 1 -- 172.21.15.106:0/969524407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa08c004d50 msgr2=0x7fa08c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a8089640 1 --2- 172.21.15.106:0/969524407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa08c004d50 0x7fa08c005160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa098009fc0 tx=0x7fa09801ccc0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a6626640 1 -- 172.21.15.106:0/969524407 reap_dead start 2024-04-25T11:20:26.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a8089640 1 -- 172.21.15.106:0/969524407 shutdown_connections 2024-04-25T11:20:26.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a8089640 1 -- 172.21.15.106:0/969524407 >> 172.21.15.106:0/969524407 conn(0x7fa0a00cffd0 msgr2=0x7fa0a00d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:26.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a8089640 1 -- 172.21.15.106:0/969524407 shutdown_connections 2024-04-25T11:20:26.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a8089640 1 -- 172.21.15.106:0/969524407 wait complete. 2024-04-25T11:20:26.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a8089640 1 Processor -- start 2024-04-25T11:20:26.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.731+0000 7fa0a8089640 1 -- start start 2024-04-25T11:20:26.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0a00c7420 0x7fa0a01d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a01d74f0 0x7fa0a01d7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0a01d7e40 0x7fa0a01d42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a00d1a20 con 0x7fa0a01d7e40 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a00b7bc0 con 0x7fa0a00c7420 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a00d1ba0 con 0x7fa0a01d74f0 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0a01d7e40 0x7fa0a01d42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0a01d7e40 0x7fa0a01d42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57302/0 (socket says 172.21.15.106:57302) 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 -- 172.21.15.106:0/796996273 learned_addr learned my addr 172.21.15.106:0/796996273 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a5e25640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0a00c7420 0x7fa0a01d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a5624640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a01d74f0 0x7fa0a01d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 -- 172.21.15.106:0/796996273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a01d74f0 msgr2=0x7fa0a01d7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a01d74f0 0x7fa0a01d7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 -- 172.21.15.106:0/796996273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0a00c7420 msgr2=0x7fa0a01d8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0a00c7420 0x7fa0a01d8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 -- 172.21.15.106:0/796996273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a01d9450 con 0x7fa0a01d7e40 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a6626640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0a01d7e40 0x7fa0a01d42b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa0980b0040 tx=0x7fa098009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0967fc640 1 -- 172.21.15.106:0/796996273 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980bb070 con 0x7fa0a01d7e40 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0967fc640 1 -- 172.21.15.106:0/796996273 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0980039a0 con 0x7fa0a01d7e40 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a01d96f0 con 0x7fa0a01d7e40 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0967fc640 1 -- 172.21.15.106:0/796996273 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980b2040 con 0x7fa0a01d7e40 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a01d9cb0 con 0x7fa0a01d7e40 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a00b8810 con 0x7fa0a01d7e40 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0967fc640 1 -- 172.21.15.106:0/796996273 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa0980be070 con 0x7fa0a01d7e40 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0967fc640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa08406f400 0x7fa084071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0967fc640 1 -- 172.21.15.106:0/796996273 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa098128890 con 0x7fa0a01d7e40 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.735+0000 7fa0a5e25640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa08406f400 0x7fa084071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.739+0000 7fa0a5e25640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa08406f400 0x7fa084071910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0a01ea9b0 tx=0x7fa09c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.739+0000 7fa0967fc640 1 -- 172.21.15.106:0/796996273 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa0980169d0 con 0x7fa0a01d7e40 2024-04-25T11:20:26.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.955+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fa0a006fb40 con 0x7fa0a01d7e40 2024-04-25T11:20:26.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.959+0000 7fa0967fc640 1 -- 172.21.15.106:0/796996273 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fa0980f5680 con 0x7fa0a01d7e40 2024-04-25T11:20:26.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa08406f400 msgr2=0x7fa084071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa08406f400 0x7fa084071910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0a01ea9b0 tx=0x7fa09c008040 comp rx=0 tx=0).stop 2024-04-25T11:20:26.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0a01d7e40 msgr2=0x7fa0a01d42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0a01d7e40 0x7fa0a01d42b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa0980b0040 tx=0x7fa098009750 comp rx=0 tx=0).stop 2024-04-25T11:20:26.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 shutdown_connections 2024-04-25T11:20:26.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa08406f400 0x7fa084071910 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0a01d7e40 0x7fa0a01d42b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0a01d74f0 0x7fa0a01d7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 --2- 172.21.15.106:0/796996273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0a00c7420 0x7fa0a01d8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 >> 172.21.15.106:0/796996273 conn(0x7fa0a00cffd0 msgr2=0x7fa0a00b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:26.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 shutdown_connections 2024-04-25T11:20:26.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:26.963+0000 7fa0a8089640 1 -- 172.21.15.106:0/796996273 wait complete. 2024-04-25T11:20:26.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T11:20:27.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedf0bb640 1 Processor -- start 2024-04-25T11:20:27.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedf0bb640 1 -- start start 2024-04-25T11:20:27.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedf0bb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fded80cf0e0 0x7fded809bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedf0bb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded809c140 0x7fded80d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedf0bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded80b46f0 con 0x7fded80d1680 2024-04-25T11:20:27.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedf0bb640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fded809ce70 con 0x7fded80cf0e0 2024-04-25T11:20:27.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedf0bb640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fded80b8ce0 con 0x7fded809c140 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedd658640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fded80d1680 0x7fded80b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34434/0 (socket says 172.21.15.106:34434) 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedd658640 1 -- 172.21.15.106:0/1340723754 learned_addr learned my addr 172.21.15.106:0/1340723754 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865481118 0 0) 0x7fded80b46f0 con 0x7fded80d1680 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedce57640 1 --2- 172.21.15.106:0/1340723754 >> v2:172.21.15.134:3300/0 conn(0x7fded80cf0e0 0x7fded809bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec0003530 con 0x7fded80d1680 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecffff640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded809c140 0x7fded80d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1067406610 0 0) 0x7fdec0003530 con 0x7fded80d1680 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded809c140 msgr2=0x7fded80d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded809c140 0x7fded80d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 >> v2:172.21.15.134:3300/0 conn(0x7fded80cf0e0 msgr2=0x7fded809bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 --2- 172.21.15.106:0/1340723754 >> v2:172.21.15.134:3300/0 conn(0x7fded80cf0e0 0x7fded809bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded80cf4f0 con 0x7fded80d1680 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fded0001a20 con 0x7fded80d1680 2024-04-25T11:20:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 >> v1:172.21.15.106:6789/0 conn(0x7fded80d1680 legacy=0x7fded80b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fdec0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec0004810 0x7fdec0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded809c140 0x7fdec0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fded809ce70 con 0x7fdec00039c0 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fded80b8ce0 con 0x7fdec0004810 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdec000ba80 con 0x7fded809c140 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedce57640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fdec0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedce57640 1 -- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded809c140 msgr2=0x7fdec0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedce57640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded809c140 0x7fdec0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedce57640 1 -- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec0004810 msgr2=0x7fdec0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedce57640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec0004810 0x7fdec0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.123+0000 7fdedce57640 1 -- 172.21.15.106:0/1340723754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec0003530 con 0x7fdec00039c0 2024-04-25T11:20:27.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedce57640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fdec0003e30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fded4009a90 tx=0x7fded401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4171b7d41c2a53b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4009690 con 0x7fdec00039c0 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded40059a0 con 0x7fdec00039c0 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdecf7fe640 1 -- 172.21.15.106:0/1340723754 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4005d50 con 0x7fdec00039c0 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedf0bb640 1 -- 172.21.15.106:0/1340723754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 msgr2=0x7fdec0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedf0bb640 1 --2- 172.21.15.106:0/1340723754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fdec0003e30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fded4009a90 tx=0x7fded401ca70 comp rx=0 tx=0).stop 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedd658640 1 -- 172.21.15.106:0/1340723754 reap_dead start 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedf0bb640 1 -- 172.21.15.106:0/1340723754 shutdown_connections 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedf0bb640 1 -- 172.21.15.106:0/1340723754 >> 172.21.15.106:0/1340723754 conn(0x7fded80b5060 msgr2=0x7fded80b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedf0bb640 1 -- 172.21.15.106:0/1340723754 shutdown_connections 2024-04-25T11:20:27.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedf0bb640 1 -- 172.21.15.106:0/1340723754 wait complete. 2024-04-25T11:20:27.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedf0bb640 1 Processor -- start 2024-04-25T11:20:27.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.127+0000 7fdedf0bb640 1 -- start start 2024-04-25T11:20:27.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fded815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fded81f73a0 0x7fded80b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded80b7b50 0x7fded80c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fded809c550 con 0x7fdec00039c0 2024-04-25T11:20:27.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fded809c870 con 0x7fded81f73a0 2024-04-25T11:20:27.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fded80b8e80 con 0x7fded80b7b50 2024-04-25T11:20:27.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fded815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fded815f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57314/0 (socket says 172.21.15.106:57314) 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 -- 172.21.15.106:0/628858984 learned_addr learned my addr 172.21.15.106:0/628858984 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedd658640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded80b7b50 0x7fded80c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdecffff640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fded81f73a0 0x7fded80b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 -- 172.21.15.106:0/628858984 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded80b7b50 msgr2=0x7fded80c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded80b7b50 0x7fded80c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 -- 172.21.15.106:0/628858984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fded81f73a0 msgr2=0x7fded80b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fded81f73a0 0x7fded80b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 -- 172.21.15.106:0/628858984 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded80c8df0 con 0x7fdec00039c0 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedce57640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fded815f690 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fded40af040 tx=0x7fdec0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdecd7fa640 1 -- 172.21.15.106:0/628858984 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4006440 con 0x7fdec00039c0 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdecd7fa640 1 -- 172.21.15.106:0/628858984 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded40bf070 con 0x7fdec00039c0 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded80c90d0 con 0x7fdec00039c0 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdecd7fa640 1 -- 172.21.15.106:0/628858984 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4006820 con 0x7fdec00039c0 2024-04-25T11:20:27.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded80c9690 con 0x7fdec00039c0 2024-04-25T11:20:27.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded8050750 con 0x7fdec00039c0 2024-04-25T11:20:27.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdecd7fa640 1 -- 172.21.15.106:0/628858984 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fded40a5550 con 0x7fdec00039c0 2024-04-25T11:20:27.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.131+0000 7fdecd7fa640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdec406f400 0x7fdec4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.135+0000 7fdecd7fa640 1 -- 172.21.15.106:0/628858984 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fded4129a00 con 0x7fdec00039c0 2024-04-25T11:20:27.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.135+0000 7fdecffff640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdec406f400 0x7fdec4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.135+0000 7fdecffff640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdec406f400 0x7fdec4071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fded80c9bd0 tx=0x7fdec8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.139+0000 7fdecd7fa640 1 -- 172.21.15.106:0/628858984 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fded4016780 con 0x7fdec00039c0 2024-04-25T11:20:27.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.351+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fded80b06d0 con 0x7fdec00039c0 2024-04-25T11:20:27.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdecd7fa640 1 -- 172.21.15.106:0/628858984 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fded40f6d60 con 0x7fdec00039c0 2024-04-25T11:20:27.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdec406f400 msgr2=0x7fdec4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdec406f400 0x7fdec4071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fded80c9bd0 tx=0x7fdec8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:27.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 msgr2=0x7fded815f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fded815f690 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fded40af040 tx=0x7fdec0005840 comp rx=0 tx=0).stop 2024-04-25T11:20:27.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 shutdown_connections 2024-04-25T11:20:27.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fded80b7b50 0x7fded80c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fded81f73a0 0x7fded80b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdec406f400 0x7fdec4071910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 --2- 172.21.15.106:0/628858984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec00039c0 0x7fded815f690 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 >> 172.21.15.106:0/628858984 conn(0x7fded80b5060 msgr2=0x7fded8075800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:27.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 shutdown_connections 2024-04-25T11:20:27.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.355+0000 7fdedf0bb640 1 -- 172.21.15.106:0/628858984 wait complete. 2024-04-25T11:20:27.374 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T11:20:27.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.507+0000 7f6971bb9640 1 Processor -- start 2024-04-25T11:20:27.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.507+0000 7f6971bb9640 1 -- start start 2024-04-25T11:20:27.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f6971bb9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 0x7f696c0b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f6971bb9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f696c0d44a0 0x7f696c0d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f6971bb9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f696c0d1350 con 0x7f696c0d2150 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f6971bb9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f696c0b9270 con 0x7f696c0d44a0 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f6971bb9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f696c163cc0 con 0x7f696c15fd10 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f696c0d44a0 0x7f696c0d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f696c0d44a0 0x7f696c0d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57330/0 (socket says 172.21.15.106:57330) 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696bfff640 1 -- 172.21.15.106:0/3872554325 learned_addr learned my addr 172.21.15.106:0/3872554325 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696b7fe640 1 --2- 172.21.15.106:0/3872554325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 0x7f696c0b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696bfff640 1 -- 172.21.15.106:0/3872554325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 msgr2=0x7f696c0b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696bfff640 1 --2- 172.21.15.106:0/3872554325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 0x7f696c0b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696bfff640 1 -- 172.21.15.106:0/3872554325 >> v1:172.21.15.134:6789/0 conn(0x7f696c0d2150 legacy=0x7f696c0d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696bfff640 1 -- 172.21.15.106:0/3872554325 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c163fa0 con 0x7f696c0d44a0 2024-04-25T11:20:27.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696bfff640 1 --2- 172.21.15.106:0/3872554325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c0d48b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6964009e50 tx=0x7f696401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcddd3859b608c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696affd640 1 -- 172.21.15.106:0/3872554325 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69640a5670 con 0x7f696c0d44a0 2024-04-25T11:20:27.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696affd640 1 -- 172.21.15.106:0/3872554325 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69640a5810 con 0x7f696c0d44a0 2024-04-25T11:20:27.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.511+0000 7f696affd640 1 -- 172.21.15.106:0/3872554325 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69640a5be0 con 0x7f696c0d44a0 2024-04-25T11:20:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 -- 172.21.15.106:0/3872554325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 msgr2=0x7f696c0d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 --2- 172.21.15.106:0/3872554325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c0d48b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6964009e50 tx=0x7f696401c9f0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 -- 172.21.15.106:0/3872554325 shutdown_connections 2024-04-25T11:20:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 --2- 172.21.15.106:0/3872554325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c0d48b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 --2- 172.21.15.106:0/3872554325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 0x7f696c0b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 -- 172.21.15.106:0/3872554325 >> 172.21.15.106:0/3872554325 conn(0x7f696c06d7f0 msgr2=0x7f696c06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 -- 172.21.15.106:0/3872554325 shutdown_connections 2024-04-25T11:20:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 -- 172.21.15.106:0/3872554325 wait complete. 2024-04-25T11:20:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 Processor -- start 2024-04-25T11:20:27.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.515+0000 7f6971bb9640 1 -- start start 2024-04-25T11:20:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f696c0d2150 0x7f696c168820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 0x7f696c159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f696c163a60 con 0x7f696c0d44a0 2024-04-25T11:20:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1638e0 con 0x7f696c0d2150 2024-04-25T11:20:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f696c0b49a0 con 0x7f696c15fd10 2024-04-25T11:20:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c15c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57340/0 (socket says 172.21.15.106:57340) 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 -- 172.21.15.106:0/2941428258 learned_addr learned my addr 172.21.15.106:0/2941428258 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f696b7fe640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f696c0d2150 0x7f696c168820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f696bfff640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 0x7f696c159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 -- 172.21.15.106:0/2941428258 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 msgr2=0x7f696c159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 0x7f696c159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 -- 172.21.15.106:0/2941428258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f696c0d2150 msgr2=0x7f696c168820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f696c0d2150 0x7f696c168820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 -- 172.21.15.106:0/2941428258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c166520 con 0x7f696c0d44a0 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6963fff640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c15c960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f695400b440 tx=0x7f695400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6968ff9640 1 -- 172.21.15.106:0/2941428258 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954015020 con 0x7f696c0d44a0 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6968ff9640 1 -- 172.21.15.106:0/2941428258 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6954011890 con 0x7f696c0d44a0 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c1667c0 con 0x7f696c0d44a0 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6968ff9640 1 -- 172.21.15.106:0/2941428258 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954011c80 con 0x7f696c0d44a0 2024-04-25T11:20:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c168d60 con 0x7f696c0d44a0 2024-04-25T11:20:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696c050750 con 0x7f696c0d44a0 2024-04-25T11:20:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6968ff9640 1 -- 172.21.15.106:0/2941428258 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6954018020 con 0x7f696c0d44a0 2024-04-25T11:20:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6968ff9640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f694c06f400 0x7f694c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f696b7fe640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f694c06f400 0x7f694c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f6968ff9640 1 -- 172.21.15.106:0/2941428258 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f695408ed20 con 0x7f696c0d44a0 2024-04-25T11:20:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.519+0000 7f696b7fe640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f694c06f400 0x7f694c071910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f695c00b7c0 tx=0x7f695c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.523+0000 7f6968ff9640 1 -- 172.21.15.106:0/2941428258 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f695401f030 con 0x7f696c0d44a0 2024-04-25T11:20:27.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.743+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f696c0d3be0 con 0x7f696c0d44a0 2024-04-25T11:20:27.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.747+0000 7f6968ff9640 1 -- 172.21.15.106:0/2941428258 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f695400d040 con 0x7f696c0d44a0 2024-04-25T11:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f694c06f400 msgr2=0x7f694c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f694c06f400 0x7f694c071910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f695c00b7c0 tx=0x7f695c008040 comp rx=0 tx=0).stop 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 msgr2=0x7f696c15c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c15c960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f695400b440 tx=0x7f695400b900 comp rx=0 tx=0).stop 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 shutdown_connections 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f696c15fd10 0x7f696c159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f694c06f400 0x7f694c071910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f696c0d44a0 0x7f696c15c960 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 --2- 172.21.15.106:0/2941428258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f696c0d2150 0x7f696c168820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 >> 172.21.15.106:0/2941428258 conn(0x7f696c06d7f0 msgr2=0x7f696c15ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 shutdown_connections 2024-04-25T11:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.751+0000 7f6971bb9640 1 -- 172.21.15.106:0/2941428258 wait complete. 2024-04-25T11:20:27.771 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T11:20:27.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.915+0000 7f65af301640 1 Processor -- start 2024-04-25T11:20:27.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.915+0000 7f65af301640 1 -- start start 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65af301640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a815cae0 0x7f65a8160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65af301640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65a80b52e0 0x7f65a80cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65af301640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a80b6120 con 0x7f65a80cf6d0 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65af301640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f65a80d0020 con 0x7f65a80b52e0 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65af301640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65a80ce2a0 con 0x7f65a815cae0 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ad09d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a815cae0 0x7f65a8160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ad89e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65a80b52e0 0x7f65a80cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ac89c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f65a80cf6d0 0x7f65a80b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:35188/0 (socket says 172.21.15.106:35188) 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ac89c640 1 -- 172.21.15.106:0/3043901621 learned_addr learned my addr 172.21.15.106:0/3043901621 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:27.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ad09d640 1 -- 172.21.15.106:0/3043901621 >> v2:172.21.15.134:3300/0 conn(0x7f65a80b52e0 msgr2=0x7f65a80cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ad09d640 1 --2- 172.21.15.106:0/3043901621 >> v2:172.21.15.134:3300/0 conn(0x7f65a80b52e0 0x7f65a80cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ad09d640 1 -- 172.21.15.106:0/3043901621 >> v1:172.21.15.134:6789/0 conn(0x7f65a80cf6d0 legacy=0x7f65a80b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ad09d640 1 -- 172.21.15.106:0/3043901621 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a8070920 con 0x7f65a815cae0 2024-04-25T11:20:27.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.919+0000 7f65ad09d640 1 --2- 172.21.15.106:0/3043901621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a815cae0 0x7f65a8160fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65a40097c0 tx=0x7f65a401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaaa872898788e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f659ffff640 1 -- 172.21.15.106:0/3043901621 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40b0070 con 0x7f65a815cae0 2024-04-25T11:20:27.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f659ffff640 1 -- 172.21.15.106:0/3043901621 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a40ab600 con 0x7f65a815cae0 2024-04-25T11:20:27.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f659ffff640 1 -- 172.21.15.106:0/3043901621 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40ab9d0 con 0x7f65a815cae0 2024-04-25T11:20:27.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 -- 172.21.15.106:0/3043901621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a815cae0 msgr2=0x7f65a8160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 --2- 172.21.15.106:0/3043901621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a815cae0 0x7f65a8160fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65a40097c0 tx=0x7f65a401c410 comp rx=0 tx=0).stop 2024-04-25T11:20:27.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 -- 172.21.15.106:0/3043901621 shutdown_connections 2024-04-25T11:20:27.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 --2- 172.21.15.106:0/3043901621 >> v2:172.21.15.134:3300/0 conn(0x7f65a80b52e0 0x7f65a80cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 --2- 172.21.15.106:0/3043901621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a815cae0 0x7f65a8160fc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 -- 172.21.15.106:0/3043901621 >> 172.21.15.106:0/3043901621 conn(0x7f65a80b3560 msgr2=0x7f65a80b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:27.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 -- 172.21.15.106:0/3043901621 shutdown_connections 2024-04-25T11:20:27.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 -- 172.21.15.106:0/3043901621 wait complete. 2024-04-25T11:20:27.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 Processor -- start 2024-04-25T11:20:27.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.923+0000 7f65af301640 1 -- start start 2024-04-25T11:20:27.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a80cf6d0 0x7f65a8072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65a815cae0 0x7f65a8073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a81663a0 0x7f65a8141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f65a80b4740 con 0x7f65a815cae0 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65a80b4460 con 0x7f65a81663a0 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65a816a3a0 con 0x7f65a80cf6d0 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65a815cae0 0x7f65a8073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65a815cae0 0x7f65a8073000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57348/0 (socket says 172.21.15.106:57348) 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 -- 172.21.15.106:0/1715566602 learned_addr learned my addr 172.21.15.106:0/1715566602 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ad89e640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a81663a0 0x7f65a8141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ad09d640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a80cf6d0 0x7f65a8072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 -- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a80cf6d0 msgr2=0x7f65a8072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a80cf6d0 0x7f65a8072ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 -- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a81663a0 msgr2=0x7f65a8141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a81663a0 0x7f65a8141530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 -- 172.21.15.106:0/1715566602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a8148160 con 0x7f65a815cae0 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ad09d640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a80cf6d0 0x7f65a8072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65ac89c640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65a815cae0 0x7f65a8073000 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f659800c3c0 tx=0x7f659800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f659dffb640 1 -- 172.21.15.106:0/1715566602 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598017070 con 0x7f65a815cae0 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f659dffb640 1 -- 172.21.15.106:0/1715566602 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6598005cf0 con 0x7f65a815cae0 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f659dffb640 1 -- 172.21.15.106:0/1715566602 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65980072f0 con 0x7f65a815cae0 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a8166b10 con 0x7f65a815cae0 2024-04-25T11:20:27.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a81670d0 con 0x7f65a815cae0 2024-04-25T11:20:27.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a8166880 con 0x7f65a815cae0 2024-04-25T11:20:27.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.927+0000 7f659dffb640 1 -- 172.21.15.106:0/1715566602 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f659801a070 con 0x7f65a815cae0 2024-04-25T11:20:27.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.931+0000 7f659dffb640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f658c06f400 0x7f658c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.931+0000 7f659dffb640 1 -- 172.21.15.106:0/1715566602 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f659808e2d0 con 0x7f65a815cae0 2024-04-25T11:20:27.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.931+0000 7f65ad09d640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f658c06f400 0x7f658c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.931+0000 7f65ad09d640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f658c06f400 0x7f658c071910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f65a40ad000 tx=0x7f65a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:27.935+0000 7f659dffb640 1 -- 172.21.15.106:0/1715566602 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f659805b490 con 0x7f65a815cae0 2024-04-25T11:20:28.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.159+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f65a8050750 con 0x7f65a815cae0 2024-04-25T11:20:28.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.163+0000 7f659dffb640 1 -- 172.21.15.106:0/1715566602 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f659805ad90 con 0x7f65a815cae0 2024-04-25T11:20:28.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f658c06f400 msgr2=0x7f658c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f658c06f400 0x7f658c071910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f65a40ad000 tx=0x7f65a4008040 comp rx=0 tx=0).stop 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65a815cae0 msgr2=0x7f65a8073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65a815cae0 0x7f65a8073000 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f659800c3c0 tx=0x7f659800c880 comp rx=0 tx=0).stop 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 shutdown_connections 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a81663a0 0x7f65a8141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f658c06f400 0x7f658c071910 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65a815cae0 0x7f65a8073000 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 --2- 172.21.15.106:0/1715566602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65a80cf6d0 0x7f65a8072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 >> 172.21.15.106:0/1715566602 conn(0x7f65a80b3560 msgr2=0x7f65a81602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 shutdown_connections 2024-04-25T11:20:28.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.167+0000 7f65af301640 1 -- 172.21.15.106:0/1715566602 wait complete. 2024-04-25T11:20:28.187 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:28.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc819ad9640 1 Processor -- start 2024-04-25T11:20:28.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc819ad9640 1 -- start start 2024-04-25T11:20:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc819ad9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc814073460 0x7fc8140d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc819ad9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc814073870 con 0x7fc8140cef60 2024-04-25T11:20:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc819ad9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8140ce050 con 0x7fc81415e0a0 2024-04-25T11:20:28.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc819ad9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc81415e6b0 con 0x7fc814073460 2024-04-25T11:20:28.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc812ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc8140cef60 0x7fc8140d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34450/0 (socket says 172.21.15.106:34450) 2024-04-25T11:20:28.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc812ffd640 1 -- 172.21.15.106:0/2183021175 learned_addr learned my addr 172.21.15.106:0/2183021175 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:28.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc813fff640 1 --2- 172.21.15.106:0/2183021175 >> v2:172.21.15.106:3300/0 conn(0x7fc814073460 0x7fc8140d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc813fff640 1 -- 172.21.15.106:0/2183021175 >> v1:172.21.15.134:6789/0 conn(0x7fc81415e0a0 legacy=0x7fc8140ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc813fff640 1 -- 172.21.15.106:0/2183021175 >> v1:172.21.15.106:6789/0 conn(0x7fc8140cef60 legacy=0x7fc8140d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc8127fc640 1 -- 172.21.15.106:0/2183021175 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3639219307 0 0) 0x7fc814073870 con 0x7fc8140cef60 2024-04-25T11:20:28.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.339+0000 7fc813fff640 1 -- 172.21.15.106:0/2183021175 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81415e990 con 0x7fc814073460 2024-04-25T11:20:28.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc813fff640 1 --2- 172.21.15.106:0/2183021175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc8140d13f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc804009d90 tx=0x7fc80401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6dc75ab8c201d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc8127fc640 1 -- 172.21.15.106:0/2183021175 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040b2070 con 0x7fc814073460 2024-04-25T11:20:28.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc8127fc640 1 -- 172.21.15.106:0/2183021175 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8040a5830 con 0x7fc814073460 2024-04-25T11:20:28.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc8127fc640 1 -- 172.21.15.106:0/2183021175 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040a5c00 con 0x7fc814073460 2024-04-25T11:20:28.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc819ad9640 1 -- 172.21.15.106:0/2183021175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 msgr2=0x7fc8140d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc819ad9640 1 --2- 172.21.15.106:0/2183021175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc8140d13f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc804009d90 tx=0x7fc80401c970 comp rx=0 tx=0).stop 2024-04-25T11:20:28.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc819ad9640 1 -- 172.21.15.106:0/2183021175 shutdown_connections 2024-04-25T11:20:28.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc819ad9640 1 --2- 172.21.15.106:0/2183021175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc8140d13f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc819ad9640 1 -- 172.21.15.106:0/2183021175 >> 172.21.15.106:0/2183021175 conn(0x7fc81406c7c0 msgr2=0x7fc81406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:28.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc819ad9640 1 -- 172.21.15.106:0/2183021175 shutdown_connections 2024-04-25T11:20:28.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.343+0000 7fc819ad9640 1 -- 172.21.15.106:0/2183021175 wait complete. 2024-04-25T11:20:28.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 Processor -- start 2024-04-25T11:20:28.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 -- start start 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc81415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81415e0a0 0x7fc814139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc81415cf30 0x7fc81413a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc81415f3f0 con 0x7fc814073460 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc81415f270 con 0x7fc81415e0a0 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc81416bf10 con 0x7fc81415cf30 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc81415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc81415c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57358/0 (socket says 172.21.15.106:57358) 2024-04-25T11:20:28.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 -- 172.21.15.106:0/11205066 learned_addr learned my addr 172.21.15.106:0/11205066 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc812ffd640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81415e0a0 0x7fc814139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 -- 172.21.15.106:0/11205066 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc81415cf30 msgr2=0x7fc81413a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc813fff640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc81415cf30 0x7fc81413a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc81415cf30 0x7fc81413a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 -- 172.21.15.106:0/11205066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81415e0a0 msgr2=0x7fc814139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81415e0a0 0x7fc814139bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 -- 172.21.15.106:0/11205066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc814151dc0 con 0x7fc814073460 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc8137fe640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc81415c9f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00c3c0 tx=0x7fc80c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc80bfff640 1 -- 172.21.15.106:0/11205066 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c017070 con 0x7fc814073460 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc80bfff640 1 -- 172.21.15.106:0/11205066 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc80c0052d0 con 0x7fc814073460 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8141520a0 con 0x7fc814073460 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc80bfff640 1 -- 172.21.15.106:0/11205066 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c003840 con 0x7fc814073460 2024-04-25T11:20:28.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc814152660 con 0x7fc814073460 2024-04-25T11:20:28.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.347+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc814075e40 con 0x7fc814073460 2024-04-25T11:20:28.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.351+0000 7fc80bfff640 1 -- 172.21.15.106:0/11205066 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc80c01a070 con 0x7fc814073460 2024-04-25T11:20:28.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.351+0000 7fc80bfff640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc7f406f400 0x7fc7f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.351+0000 7fc80bfff640 1 -- 172.21.15.106:0/11205066 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fc80c08e310 con 0x7fc814073460 2024-04-25T11:20:28.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.351+0000 7fc812ffd640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc7f406f400 0x7fc7f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.351+0000 7fc812ffd640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc7f406f400 0x7fc7f4071910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc81406ea40 tx=0x7fc7fc007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.355+0000 7fc80bfff640 1 -- 172.21.15.106:0/11205066 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc80c005480 con 0x7fc814073460 2024-04-25T11:20:28.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.503+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc81406da30 con 0x7fc814073460 2024-04-25T11:20:28.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc80bfff640 1 -- 172.21.15.106:0/11205066 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12927 (secure 0 0 0) 0x7fc80c05b560 con 0x7fc814073460 2024-04-25T11:20:28.506 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:28.507 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":29,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:25.391094+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc7f406f400 msgr2=0x7fc7f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc7f406f400 0x7fc7f4071910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc81406ea40 tx=0x7fc7fc007d30 comp rx=0 tx=0).stop 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 msgr2=0x7fc81415c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc81415c9f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00c3c0 tx=0x7fc80c00c880 comp rx=0 tx=0).stop 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 shutdown_connections 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc81415cf30 0x7fc81413a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81415e0a0 0x7fc814139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc7f406f400 0x7fc7f4071910 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 --2- 172.21.15.106:0/11205066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc814073460 0x7fc81415c9f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 >> 172.21.15.106:0/11205066 conn(0x7fc81406c7c0 msgr2=0x7fc8140b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 shutdown_connections 2024-04-25T11:20:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.507+0000 7fc819ad9640 1 -- 172.21.15.106:0/11205066 wait complete. 2024-04-25T11:20:28.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:28.526 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:28.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa73e38c640 1 Processor -- start 2024-04-25T11:20:28.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa73e38c640 1 -- start start 2024-04-25T11:20:28.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa73e38c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 0x7fa7380d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa73e38c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7380cdd40 con 0x7fa7380d1c80 2024-04-25T11:20:28.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa73e38c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7380d2090 con 0x7fa738143fa0 2024-04-25T11:20:28.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa73e38c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa738073300 con 0x7fa7380cd930 2024-04-25T11:20:28.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa73c929640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa7380d1c80 0x7fa7380b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34464/0 (socket says 172.21.15.106:34464) 2024-04-25T11:20:28.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa73c929640 1 -- 172.21.15.106:0/2285641040 learned_addr learned my addr 172.21.15.106:0/2285641040 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:28.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa7377fe640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 0x7fa7380d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044273104 0 0) 0x7fa730001120 con 0x7fa7380d1c80 2024-04-25T11:20:28.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa724003530 con 0x7fa7380d1c80 2024-04-25T11:20:28.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1072842960 0 0) 0x7fa724003530 con 0x7fa7380d1c80 2024-04-25T11:20:28.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 msgr2=0x7fa7380d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 0x7fa7380d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 >> v1:172.21.15.134:6789/0 conn(0x7fa738143fa0 legacy=0x7fa738072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7380735e0 con 0x7fa7380d1c80 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fa730001850 con 0x7fa7380d1c80 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 >> v1:172.21.15.106:6789/0 conn(0x7fa7380d1c80 legacy=0x7fa7380b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7240039f0 0x7fa724003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa724004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.671+0000 7fa736ffd640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 0x7fa724005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa738073300 con 0x7fa724004810 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa72400bb00 con 0x7fa7240039f0 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa72400bd00 con 0x7fa7380cd930 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa7377fe640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa724004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73c929640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 0x7fa724005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa737fff640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7240039f0 0x7fa724003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa7377fe640 1 -- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 msgr2=0x7fa724005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa7377fe640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 0x7fa724005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa7377fe640 1 -- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7240039f0 msgr2=0x7fa724003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa7377fe640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7240039f0 0x7fa724003e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa7377fe640 1 -- 172.21.15.106:0/2285641040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa724003530 con 0x7fa724004810 2024-04-25T11:20:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73c929640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380cd930 0x7fa724005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:28.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa7377fe640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa724004c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa720009ab0 tx=0x7fa72001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c3f540f1288323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7200b2070 con 0x7fa724004810 2024-04-25T11:20:28.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7200a5550 con 0x7fa724004810 2024-04-25T11:20:28.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa736ffd640 1 -- 172.21.15.106:0/2285641040 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7200096b0 con 0x7fa724004810 2024-04-25T11:20:28.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73e38c640 1 -- 172.21.15.106:0/2285641040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 msgr2=0x7fa724004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73e38c640 1 --2- 172.21.15.106:0/2285641040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa724004c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa720009ab0 tx=0x7fa72001ca70 comp rx=0 tx=0).stop 2024-04-25T11:20:28.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73c929640 1 -- 172.21.15.106:0/2285641040 reap_dead start 2024-04-25T11:20:28.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73e38c640 1 -- 172.21.15.106:0/2285641040 shutdown_connections 2024-04-25T11:20:28.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73e38c640 1 -- 172.21.15.106:0/2285641040 >> 172.21.15.106:0/2285641040 conn(0x7fa73806c9f0 msgr2=0x7fa73806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:28.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73e38c640 1 -- 172.21.15.106:0/2285641040 shutdown_connections 2024-04-25T11:20:28.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.675+0000 7fa73e38c640 1 -- 172.21.15.106:0/2285641040 wait complete. 2024-04-25T11:20:28.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 Processor -- start 2024-04-25T11:20:28.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 -- start start 2024-04-25T11:20:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa73815c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380704a0 0x7fa738145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738145a70 0x7fa73806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380b5320 con 0x7fa724004810 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380b5040 con 0x7fa7380704a0 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa7380d1740 con 0x7fa738145a70 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa73815c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa73815c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57376/0 (socket says 172.21.15.106:57376) 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 -- 172.21.15.106:0/4070206527 learned_addr learned my addr 172.21.15.106:0/4070206527 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73c929640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738145a70 0x7fa73806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa7377fe640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380704a0 0x7fa738145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 -- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738145a70 msgr2=0x7fa73806ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738145a70 0x7fa73806ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 -- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380704a0 msgr2=0x7fa738145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380704a0 0x7fa738145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 -- 172.21.15.106:0/4070206527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738145e80 con 0x7fa724004810 2024-04-25T11:20:28.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa7377fe640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380704a0 0x7fa738145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:28.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa737fff640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa73815c2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa72800d900 tx=0x7fa72800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa734ff9640 1 -- 172.21.15.106:0/4070206527 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728003db0 con 0x7fa724004810 2024-04-25T11:20:28.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa734ff9640 1 -- 172.21.15.106:0/4070206527 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa728017070 con 0x7fa724004810 2024-04-25T11:20:28.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73815f6d0 con 0x7fa724004810 2024-04-25T11:20:28.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa734ff9640 1 -- 172.21.15.106:0/4070206527 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72800b970 con 0x7fa724004810 2024-04-25T11:20:28.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.679+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73815fc90 con 0x7fa724004810 2024-04-25T11:20:28.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.683+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa738050750 con 0x7fa724004810 2024-04-25T11:20:28.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.683+0000 7fa734ff9640 1 -- 172.21.15.106:0/4070206527 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa72800bb10 con 0x7fa724004810 2024-04-25T11:20:28.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.683+0000 7fa734ff9640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa71806f430 0x7fa718071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.683+0000 7fa734ff9640 1 -- 172.21.15.106:0/4070206527 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa72808f220 con 0x7fa724004810 2024-04-25T11:20:28.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.683+0000 7fa7377fe640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa71806f430 0x7fa718071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.683+0000 7fa7377fe640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa71806f430 0x7fa718071940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7200af000 tx=0x7fa7200046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.687+0000 7fa734ff9640 1 -- 172.21.15.106:0/4070206527 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa738145e80 con 0x7fa724004810 2024-04-25T11:20:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.835+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa7380b4e20 con 0x7fa724004810 2024-04-25T11:20:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa734ff9640 1 -- 172.21.15.106:0/4070206527 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fa7380d2090 con 0x7fa724004810 2024-04-25T11:20:28.838 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa71806f430 msgr2=0x7fa718071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa71806f430 0x7fa718071940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7200af000 tx=0x7fa7200046d0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 msgr2=0x7fa73815c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa73815c2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa72800d900 tx=0x7fa72800ddc0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 shutdown_connections 2024-04-25T11:20:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738145a70 0x7fa73806ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380704a0 0x7fa738145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa71806f430 0x7fa718071940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 --2- 172.21.15.106:0/4070206527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa724004810 0x7fa73815c2f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 >> 172.21.15.106:0/4070206527 conn(0x7fa73806c9f0 msgr2=0x7fa738141630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 shutdown_connections 2024-04-25T11:20:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.839+0000 7fa73e38c640 1 -- 172.21.15.106:0/4070206527 wait complete. 2024-04-25T11:20:28.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:20:28.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f05245e2640 1 Processor -- start 2024-04-25T11:20:28.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f05245e2640 1 -- start start 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f05245e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0b43f0 0x7f051c072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f05245e2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0733a0 con 0x7f051c072f90 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f05245e2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0b49b0 con 0x7f051c15c270 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f05245e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f051c15fdb0 con 0x7f051c0b43f0 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f0522b7f640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f051c072f90 0x7f051c160240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34466/0 (socket says 172.21.15.106:34466) 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f0522b7f640 1 -- 172.21.15.106:0/4013479365 learned_addr learned my addr 172.21.15.106:0/4013479365 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f0521b7d640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0b43f0 0x7f051c072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 714675284 0 0) 0x7f050c0014a0 con 0x7f051c072f90 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0504004640 con 0x7f051c072f90 2024-04-25T11:20:29.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1605054879 0 0) 0x7f0504004640 con 0x7f051c072f90 2024-04-25T11:20:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0b43f0 msgr2=0x7f051c072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0b43f0 0x7f051c072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 >> v1:172.21.15.134:6789/0 conn(0x7f051c15c270 legacy=0x7f051c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c0b4c90 con 0x7f051c072f90 2024-04-25T11:20:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f050c001bd0 con 0x7f051c072f90 2024-04-25T11:20:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 >> v1:172.21.15.106:6789/0 conn(0x7f051c072f90 legacy=0x7f051c160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0b43f0 0x7f0504004b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05040050d0 0x7f0504005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504005e60 0x7f05040062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f051c15fdb0 con 0x7f05040050d0 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f050400ccd0 con 0x7f0504005e60 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f050400ced0 con 0x7f051c0b43f0 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:28.999+0000 7f0521b7d640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05040050d0 0x7f0504005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f0522b7f640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504005e60 0x7f05040062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f052237e640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0b43f0 0x7f0504004b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f0521b7d640 1 -- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0b43f0 msgr2=0x7f0504004b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:29.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f0521b7d640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0b43f0 0x7f0504004b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f0521b7d640 1 -- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504005e60 msgr2=0x7f05040062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f0521b7d640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504005e60 0x7f05040062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f0521b7d640 1 -- 172.21.15.106:0/4013479365 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0504004640 con 0x7f05040050d0 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f0521b7d640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05040050d0 0x7f0504005920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0510009ab0 tx=0x7f051001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3812d023d68165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100b2070 con 0x7f05040050d0 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05100a5660 con 0x7f05040050d0 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f052137c640 1 -- 172.21.15.106:0/4013479365 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100096b0 con 0x7f05040050d0 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f05245e2640 1 -- 172.21.15.106:0/4013479365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05040050d0 msgr2=0x7f0504005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f05245e2640 1 --2- 172.21.15.106:0/4013479365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05040050d0 0x7f0504005920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0510009ab0 tx=0x7f051001ca70 comp rx=0 tx=0).stop 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f0522b7f640 1 -- 172.21.15.106:0/4013479365 reap_dead start 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f05245e2640 1 -- 172.21.15.106:0/4013479365 shutdown_connections 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f05245e2640 1 -- 172.21.15.106:0/4013479365 >> 172.21.15.106:0/4013479365 conn(0x7f051c06c7c0 msgr2=0x7f051c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f05245e2640 1 -- 172.21.15.106:0/4013479365 shutdown_connections 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.003+0000 7f05245e2640 1 -- 172.21.15.106:0/4013479365 wait complete. 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 Processor -- start 2024-04-25T11:20:29.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 -- start start 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05040050d0 0x7f051c0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c1537b0 0x7f051c15d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f051c153bc0 0x7f051c15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f051c073c00 con 0x7f051c153bc0 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f051c0b5250 con 0x7f05040050d0 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f051c0cf3d0 con 0x7f051c1537b0 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f051c153bc0 0x7f051c15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f051c153bc0 0x7f051c15ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57388/0 (socket says 172.21.15.106:57388) 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 -- 172.21.15.106:0/3488065808 learned_addr learned my addr 172.21.15.106:0/3488065808 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f052237e640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05040050d0 0x7f051c0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 -- 172.21.15.106:0/3488065808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c1537b0 msgr2=0x7f051c15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c1537b0 0x7f051c15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 -- 172.21.15.106:0/3488065808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05040050d0 msgr2=0x7f051c0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05040050d0 0x7f051c0acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 -- 172.21.15.106:0/3488065808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c0ad0b0 con 0x7f051c153bc0 2024-04-25T11:20:29.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f0522b7f640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f051c153bc0 0x7f051c15ddb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f051c0b6880 tx=0x7f050c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f051affd640 1 -- 172.21.15.106:0/3488065808 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c010040 con 0x7f051c153bc0 2024-04-25T11:20:29.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f051affd640 1 -- 172.21.15.106:0/3488065808 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050c01a070 con 0x7f051c153bc0 2024-04-25T11:20:29.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c0aa5c0 con 0x7f051c153bc0 2024-04-25T11:20:29.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f051affd640 1 -- 172.21.15.106:0/3488065808 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c013ae0 con 0x7f051c153bc0 2024-04-25T11:20:29.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c0aab80 con 0x7f051c153bc0 2024-04-25T11:20:29.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.007+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051c0ad330 con 0x7f051c153bc0 2024-04-25T11:20:29.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.011+0000 7f051affd640 1 -- 172.21.15.106:0/3488065808 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f050c013c80 con 0x7f051c153bc0 2024-04-25T11:20:29.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.011+0000 7f051affd640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f050806f430 0x7f0508071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.011+0000 7f051affd640 1 -- 172.21.15.106:0/3488065808 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f050c092700 con 0x7f051c153bc0 2024-04-25T11:20:29.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.011+0000 7f052237e640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f050806f430 0x7f0508071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.011+0000 7f052237e640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f050806f430 0x7f0508071940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0514005510 tx=0x7f0514009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.015+0000 7f051affd640 1 -- 172.21.15.106:0/3488065808 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f051c0733a0 con 0x7f051c153bc0 2024-04-25T11:20:29.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.167+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f051c0b3770 con 0x7f051c153bc0 2024-04-25T11:20:29.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.167+0000 7f051affd640 1 -- 172.21.15.106:0/3488065808 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f050c05fab0 con 0x7f051c153bc0 2024-04-25T11:20:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:29.169 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-25T11:20:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T11:20:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-25T11:20:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-25T11:20:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:29.170 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-25T11:20:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f050806f430 msgr2=0x7f0508071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f050806f430 0x7f0508071940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0514005510 tx=0x7f0514009040 comp rx=0 tx=0).stop 2024-04-25T11:20:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f051c153bc0 msgr2=0x7f051c15ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f051c153bc0 0x7f051c15ddb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f051c0b6880 tx=0x7f050c0119a0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 shutdown_connections 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f050806f430 0x7f0508071940 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f051c153bc0 0x7f051c15ddb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c1537b0 0x7f051c15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 --2- 172.21.15.106:0/3488065808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05040050d0 0x7f051c0acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 >> 172.21.15.106:0/3488065808 conn(0x7f051c06c7c0 msgr2=0x7f051c157630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 shutdown_connections 2024-04-25T11:20:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.171+0000 7f05245e2640 1 -- 172.21.15.106:0/3488065808 wait complete. 2024-04-25T11:20:29.189 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:20:29.189 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:20:29.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29df8ee640 1 Processor -- start 2024-04-25T11:20:29.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29df8ee640 1 -- start start 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29df8ee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f29d80d0310 0x7f29d80bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29df8ee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80b8c10 0x7f29d80b9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29df8ee640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d80bb280 con 0x7f29d80d1f30 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29df8ee640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f29d80b7780 con 0x7f29d80d0310 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29df8ee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f29d80b7a60 con 0x7f29d80b8c10 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29dd68a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f29d80d0310 0x7f29d80bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29dde8b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f29d80d1f30 0x7f29d80d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34482/0 (socket says 172.21.15.106:34482) 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29dde8b640 1 -- 172.21.15.106:0/2989711228 learned_addr learned my addr 172.21.15.106:0/2989711228 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:29.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29dce89640 1 --2- 172.21.15.106:0/2989711228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80b8c10 0x7f29d80b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29dd68a640 1 -- 172.21.15.106:0/2989711228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80b8c10 msgr2=0x7f29d80b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:29.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29dd68a640 1 --2- 172.21.15.106:0/2989711228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80b8c10 0x7f29d80b9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29dd68a640 1 -- 172.21.15.106:0/2989711228 >> v1:172.21.15.106:6789/0 conn(0x7f29d80d1f30 legacy=0x7f29d80d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29dd68a640 1 -- 172.21.15.106:0/2989711228 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d80b7d40 con 0x7f29d80d0310 2024-04-25T11:20:29.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.327+0000 7f29cffff640 1 -- 172.21.15.106:0/2989711228 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1812284943 0 0) 0x7f29d80bb280 con 0x7f29d80d1f30 2024-04-25T11:20:29.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29dd68a640 1 --2- 172.21.15.106:0/2989711228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80bad40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f29c80097c0 tx=0x7f29c801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f187f64e65d296 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29cffff640 1 -- 172.21.15.106:0/2989711228 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80b1070 con 0x7f29d80d0310 2024-04-25T11:20:29.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29cffff640 1 -- 172.21.15.106:0/2989711228 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c80a8050 con 0x7f29d80d0310 2024-04-25T11:20:29.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29cffff640 1 -- 172.21.15.106:0/2989711228 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80ac860 con 0x7f29d80d0310 2024-04-25T11:20:29.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29df8ee640 1 -- 172.21.15.106:0/2989711228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 msgr2=0x7f29d80bad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29df8ee640 1 --2- 172.21.15.106:0/2989711228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80bad40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f29c80097c0 tx=0x7f29c801c3e0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29df8ee640 1 -- 172.21.15.106:0/2989711228 shutdown_connections 2024-04-25T11:20:29.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29df8ee640 1 --2- 172.21.15.106:0/2989711228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80b8c10 0x7f29d80b9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29df8ee640 1 --2- 172.21.15.106:0/2989711228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80bad40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29df8ee640 1 -- 172.21.15.106:0/2989711228 >> 172.21.15.106:0/2989711228 conn(0x7f29d806d7d0 msgr2=0x7f29d806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:29.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29df8ee640 1 -- 172.21.15.106:0/2989711228 shutdown_connections 2024-04-25T11:20:29.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.331+0000 7f29df8ee640 1 -- 172.21.15.106:0/2989711228 wait complete. 2024-04-25T11:20:29.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 Processor -- start 2024-04-25T11:20:29.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 -- start start 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d80b8c10 0x7f29d81665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80d1f30 0x7f29d8146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29d81645b0 con 0x7f29d80d0310 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29d8164730 con 0x7f29d80b8c10 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f29d815f400 con 0x7f29d80d1f30 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80cf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57412/0 (socket says 172.21.15.106:57412) 2024-04-25T11:20:29.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 -- 172.21.15.106:0/840503425 learned_addr learned my addr 172.21.15.106:0/840503425 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dde8b640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80d1f30 0x7f29d8146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dd68a640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d80b8c10 0x7f29d81665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 -- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80d1f30 msgr2=0x7f29d8146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80d1f30 0x7f29d8146750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 -- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d80b8c10 msgr2=0x7f29d81665e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d80b8c10 0x7f29d81665e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 -- 172.21.15.106:0/840503425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d8166b20 con 0x7f29d80d0310 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dde8b640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80d1f30 0x7f29d8146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dd68a640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d80b8c10 0x7f29d81665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29dce89640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80cf370 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d620 tx=0x7f29d400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29cdffb640 1 -- 172.21.15.106:0/840503425 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d400b620 con 0x7f29d80d0310 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29cdffb640 1 -- 172.21.15.106:0/840503425 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d4019070 con 0x7f29d80d0310 2024-04-25T11:20:29.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29cdffb640 1 -- 172.21.15.106:0/840503425 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d400b9e0 con 0x7f29d80d0310 2024-04-25T11:20:29.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d8165540 con 0x7f29d80d0310 2024-04-25T11:20:29.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d8165b00 con 0x7f29d80d0310 2024-04-25T11:20:29.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.335+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d81652b0 con 0x7f29d80d0310 2024-04-25T11:20:29.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.339+0000 7f29cdffb640 1 -- 172.21.15.106:0/840503425 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f29d400bb80 con 0x7f29d80d0310 2024-04-25T11:20:29.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.339+0000 7f29cdffb640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f29bc06f400 0x7f29bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.339+0000 7f29cdffb640 1 -- 172.21.15.106:0/840503425 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f29d408f420 con 0x7f29d80d0310 2024-04-25T11:20:29.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.339+0000 7f29dd68a640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f29bc06f400 0x7f29bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.339+0000 7f29dd68a640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f29bc06f400 0x7f29bc071910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f29c80ae000 tx=0x7f29c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.343+0000 7f29cdffb640 1 -- 172.21.15.106:0/840503425 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29d405c660 con 0x7f29d80d0310 2024-04-25T11:20:29.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:29.499+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f29d806ed60 con 0x7f29d80d0310 2024-04-25T11:20:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.387+0000 7f29cdffb640 1 -- 172.21.15.106:0/840503425 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f29d405bf60 con 0x7f29d80d0310 2024-04-25T11:20:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-25T11:20:30.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f29bc06f400 msgr2=0x7f29bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f29bc06f400 0x7f29bc071910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f29c80ae000 tx=0x7f29c8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 msgr2=0x7f29d80cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80cf370 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d620 tx=0x7f29d400dae0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 shutdown_connections 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d80d1f30 0x7f29d8146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f29bc06f400 0x7f29bc071910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29d80d0310 0x7f29d80cf370 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 --2- 172.21.15.106:0/840503425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d80b8c10 0x7f29d81665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 >> 172.21.15.106:0/840503425 conn(0x7f29d806d7d0 msgr2=0x7f29d80b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 shutdown_connections 2024-04-25T11:20:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.391+0000 7f29df8ee640 1 -- 172.21.15.106:0/840503425 wait complete. 2024-04-25T11:20:30.421 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:20:30.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.583+0000 7fa1ac0c7640 1 Processor -- start 2024-04-25T11:20:30.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.583+0000 7fa1ac0c7640 1 -- start start 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1ac0c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 0x7fa1a4160600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1ac0c7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1a40d3fe0 0x7fa1a40d43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1ac0c7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a40d0650 con 0x7fa1a40d1f60 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1ac0c7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a41647d0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1ac0c7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a4160e90 con 0x7fa1a41601f0 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1a9662640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa1a40d1f60 0x7fa1a40d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34490/0 (socket says 172.21.15.106:34490) 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1aa664640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1a40d3fe0 0x7fa1a40d43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1a9662640 1 -- 172.21.15.106:0/3054826544 learned_addr learned my addr 172.21.15.106:0/3054826544 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1a9e63640 1 --2- 172.21.15.106:0/3054826544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 0x7fa1a4160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1aa664640 1 -- 172.21.15.106:0/3054826544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 msgr2=0x7fa1a4160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1aa664640 1 --2- 172.21.15.106:0/3054826544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 0x7fa1a4160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1aa664640 1 -- 172.21.15.106:0/3054826544 >> v1:172.21.15.106:6789/0 conn(0x7fa1a40d1f60 legacy=0x7fa1a40d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1a8e61640 1 -- 172.21.15.106:0/3054826544 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3426393150 0 0) 0x7fa1a40d0650 con 0x7fa1a40d1f60 2024-04-25T11:20:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.587+0000 7fa1aa664640 1 -- 172.21.15.106:0/3054826544 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a41638d0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1aa664640 1 --2- 172.21.15.106:0/3054826544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a40d43f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4164950 tx=0x7fa19c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1130dcbef9a4b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1a8e61640 1 -- 172.21.15.106:0/3054826544 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0b2070 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1a8e61640 1 -- 172.21.15.106:0/3054826544 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c0a58f0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1a8e61640 1 -- 172.21.15.106:0/3054826544 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0a5cc0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/3054826544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 msgr2=0x7fa1a40d43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/3054826544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a40d43f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4164950 tx=0x7fa19c01c920 comp rx=0 tx=0).stop 2024-04-25T11:20:30.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/3054826544 shutdown_connections 2024-04-25T11:20:30.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/3054826544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a40d43f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/3054826544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 0x7fa1a4160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/3054826544 >> 172.21.15.106:0/3054826544 conn(0x7fa1a406d830 msgr2=0x7fa1a406dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:30.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/3054826544 shutdown_connections 2024-04-25T11:20:30.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/3054826544 wait complete. 2024-04-25T11:20:30.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 Processor -- start 2024-04-25T11:20:30.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.591+0000 7fa1ac0c7640 1 -- start start 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1a40d1f60 0x7fa1a4153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a4166b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 0x7fa1a41690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a415da70 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4160b40 con 0x7fa1a40d1f60 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a415dbf0 con 0x7fa1a41601f0 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a4166b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a4166b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57422/0 (socket says 172.21.15.106:57422) 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 -- 172.21.15.106:0/1015774006 learned_addr learned my addr 172.21.15.106:0/1015774006 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9e63640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1a40d1f60 0x7fa1a4153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1aa664640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 0x7fa1a41690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 -- 172.21.15.106:0/1015774006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 msgr2=0x7fa1a41690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 0x7fa1a41690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 -- 172.21.15.106:0/1015774006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1a40d1f60 msgr2=0x7fa1a4153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1a40d1f60 0x7fa1a4153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 -- 172.21.15.106:0/1015774006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a41685f0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1a9662640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a4166b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c480 tx=0x7fa19400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa19a7fc640 1 -- 172.21.15.106:0/1015774006 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194018070 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa19a7fc640 1 -- 172.21.15.106:0/1015774006 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1940074e0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a416a440 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa19a7fc640 1 -- 172.21.15.106:0/1015774006 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940049f0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41688b0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.595+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a40d3b90 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.599+0000 7fa19a7fc640 1 -- 172.21.15.106:0/1015774006 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa19401b070 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.599+0000 7fa19a7fc640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa18806f400 0x7fa188071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.599+0000 7fa19a7fc640 1 -- 172.21.15.106:0/1015774006 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7fa19408e6c0 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.599+0000 7fa1a9e63640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa18806f400 0x7fa188071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.599+0000 7fa1a9e63640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa18806f400 0x7fa188071910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40d3b60 tx=0x7fa1a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.603+0000 7fa19a7fc640 1 -- 172.21.15.106:0/1015774006 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa194022080 con 0x7fa1a40d3fe0 2024-04-25T11:20:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:30.767+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa1a416a1d0 con 0x7fa1a40d3fe0 2024-04-25T11:20:31.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.399+0000 7fa19a7fc640 1 -- 172.21.15.106:0/1015774006 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7fa19405b900 con 0x7fa1a40d3fe0 2024-04-25T11:20:31.399 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-25T11:20:31.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.403+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa18806f400 msgr2=0x7fa188071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.403+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa18806f400 0x7fa188071910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40d3b60 tx=0x7fa1a0008040 comp rx=0 tx=0).stop 2024-04-25T11:20:31.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.403+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 msgr2=0x7fa1a4166b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.403+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a4166b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c480 tx=0x7fa19400c940 comp rx=0 tx=0).stop 2024-04-25T11:20:31.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.411+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 shutdown_connections 2024-04-25T11:20:31.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.411+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1a41601f0 0x7fa1a41690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.411+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fa18806f400 0x7fa188071910 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.411+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1a40d3fe0 0x7fa1a4166b20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.411+0000 7fa1ac0c7640 1 --2- 172.21.15.106:0/1015774006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1a40d1f60 0x7fa1a4153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.411+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 >> 172.21.15.106:0/1015774006 conn(0x7fa1a406d830 msgr2=0x7fa1a415cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:31.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.411+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 shutdown_connections 2024-04-25T11:20:31.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.411+0000 7fa1ac0c7640 1 -- 172.21.15.106:0/1015774006 wait complete. 2024-04-25T11:20:31.445 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:20:31.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f88cc5640 1 Processor -- start 2024-04-25T11:20:31.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f88cc5640 1 -- start start 2024-04-25T11:20:31.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f88cc5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6f840cf4d0 0x7f6f840d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f88cc5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6f840d0cb0 0x7f6f840d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f88cc5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 0x7f6f840b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f88cc5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6f840707b0 con 0x7f6f840d0cb0 2024-04-25T11:20:31.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f88cc5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8406f880 con 0x7f6f840cf4d0 2024-04-25T11:20:31.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f88cc5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8406ea20 con 0x7f6f8415ca10 2024-04-25T11:20:31.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f79d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6f840d0cb0 0x7f6f840d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f79d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6f840d0cb0 0x7f6f840d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57428/0 (socket says 172.21.15.106:57428) 2024-04-25T11:20:31.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f79d9b640 1 -- 172.21.15.106:0/1825573527 learned_addr learned my addr 172.21.15.106:0/1825573527 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:31.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f8259c640 1 --2- 172.21.15.106:0/1825573527 >> v2:172.21.15.134:3300/0 conn(0x7f6f840cf4d0 0x7f6f840d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f82d9d640 1 --2- 172.21.15.106:0/1825573527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 0x7f6f840b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f79d9b640 1 -- 172.21.15.106:0/1825573527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 msgr2=0x7f6f840b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:31.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f79d9b640 1 --2- 172.21.15.106:0/1825573527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 0x7f6f840b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f79d9b640 1 -- 172.21.15.106:0/1825573527 >> v2:172.21.15.134:3300/0 conn(0x7f6f840cf4d0 msgr2=0x7f6f840d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:31.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f79d9b640 1 --2- 172.21.15.106:0/1825573527 >> v2:172.21.15.134:3300/0 conn(0x7f6f840cf4d0 0x7f6f840d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.595+0000 7f6f79d9b640 1 -- 172.21.15.106:0/1825573527 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8406ec70 con 0x7f6f840d0cb0 2024-04-25T11:20:31.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f79d9b640 1 --2- 172.21.15.106:0/1825573527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840d0cb0 0x7f6f840d1c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6f84069090 tx=0x7f6f6c01e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0ded280863581f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f81d9b640 1 -- 172.21.15.106:0/1825573527 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0ab620 con 0x7f6f840d0cb0 2024-04-25T11:20:31.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f81d9b640 1 -- 172.21.15.106:0/1825573527 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f6c0ab7c0 con 0x7f6f840d0cb0 2024-04-25T11:20:31.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f81d9b640 1 -- 172.21.15.106:0/1825573527 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0abb90 con 0x7f6f840d0cb0 2024-04-25T11:20:31.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 -- 172.21.15.106:0/1825573527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840d0cb0 msgr2=0x7f6f840d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/1825573527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840d0cb0 0x7f6f840d1c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6f84069090 tx=0x7f6f6c01e6a0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 -- 172.21.15.106:0/1825573527 shutdown_connections 2024-04-25T11:20:31.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/1825573527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 0x7f6f840b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/1825573527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840d0cb0 0x7f6f840d1c80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/1825573527 >> v2:172.21.15.134:3300/0 conn(0x7f6f840cf4d0 0x7f6f840d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 -- 172.21.15.106:0/1825573527 >> 172.21.15.106:0/1825573527 conn(0x7f6f8406c9f0 msgr2=0x7f6f8406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:31.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 -- 172.21.15.106:0/1825573527 shutdown_connections 2024-04-25T11:20:31.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 -- 172.21.15.106:0/1825573527 wait complete. 2024-04-25T11:20:31.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 Processor -- start 2024-04-25T11:20:31.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.599+0000 7f6f88cc5640 1 -- start start 2024-04-25T11:20:31.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840cf4d0 0x7f6f8414c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f840d0cb0 0x7f6f84151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 0x7f6f84144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84072ba0 con 0x7f6f840cf4d0 2024-04-25T11:20:31.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84072a20 con 0x7f6f840d0cb0 2024-04-25T11:20:31.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6f84072d20 con 0x7f6f8415ca10 2024-04-25T11:20:31.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840cf4d0 0x7f6f8414c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840cf4d0 0x7f6f8414c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57430/0 (socket says 172.21.15.106:57430) 2024-04-25T11:20:31.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 -- 172.21.15.106:0/2025526886 learned_addr learned my addr 172.21.15.106:0/2025526886 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:31.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f79d9b640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f840d0cb0 0x7f6f84151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f82d9d640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 0x7f6f84144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 -- 172.21.15.106:0/2025526886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 msgr2=0x7f6f84144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 0x7f6f84144de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 -- 172.21.15.106:0/2025526886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f840d0cb0 msgr2=0x7f6f84151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f840d0cb0 0x7f6f84151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 -- 172.21.15.106:0/2025526886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f841503c0 con 0x7f6f840cf4d0 2024-04-25T11:20:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f8259c640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840cf4d0 0x7f6f8414c8d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6f74007e60 tx=0x7f6f7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f7b7fe640 1 -- 172.21.15.106:0/2025526886 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7401a070 con 0x7f6f840cf4d0 2024-04-25T11:20:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f7b7fe640 1 -- 172.21.15.106:0/2025526886 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f74003ba0 con 0x7f6f840cf4d0 2024-04-25T11:20:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f7b7fe640 1 -- 172.21.15.106:0/2025526886 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74004030 con 0x7f6f840cf4d0 2024-04-25T11:20:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f841506a0 con 0x7f6f840cf4d0 2024-04-25T11:20:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f84150c60 con 0x7f6f840cf4d0 2024-04-25T11:20:31.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f84050750 con 0x7f6f840cf4d0 2024-04-25T11:20:31.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f7b7fe640 1 -- 172.21.15.106:0/2025526886 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6f7401d070 con 0x7f6f840cf4d0 2024-04-25T11:20:31.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f7b7fe640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f6406f430 0x7f6f64071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f7b7fe640 1 -- 172.21.15.106:0/2025526886 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f6f7408f420 con 0x7f6f840cf4d0 2024-04-25T11:20:31.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.603+0000 7f6f79d9b640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f6406f430 0x7f6f64071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.607+0000 7f6f79d9b640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f6406f430 0x7f6f64071940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0b0000 tx=0x7f6f6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.611+0000 7f6f7b7fe640 1 -- 172.21.15.106:0/2025526886 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f7405bca0 con 0x7f6f840cf4d0 2024-04-25T11:20:31.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:31.807+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6f840b19e0 con 0x7f6f840cf4d0 2024-04-25T11:20:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.403+0000 7f6f7b7fe640 1 -- 172.21.15.106:0/2025526886 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f6f7405ffa0 con 0x7f6f840cf4d0 2024-04-25T11:20:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T11:20:32.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.407+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f6406f430 msgr2=0x7f6f64071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.407+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f6406f430 0x7f6f64071940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0b0000 tx=0x7f6f6c009040 comp rx=0 tx=0).stop 2024-04-25T11:20:32.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.407+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840cf4d0 msgr2=0x7f6f8414c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.407+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840cf4d0 0x7f6f8414c8d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6f74007e60 tx=0x7f6f7400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.411+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 shutdown_connections 2024-04-25T11:20:32.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.411+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f8415ca10 0x7f6f84144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.411+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f840d0cb0 0x7f6f84151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.411+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6f6406f430 0x7f6f64071940 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.411+0000 7f6f88cc5640 1 --2- 172.21.15.106:0/2025526886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f840cf4d0 0x7f6f8414c8d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.411+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 >> 172.21.15.106:0/2025526886 conn(0x7f6f8406c9f0 msgr2=0x7f6f84152c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:32.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.411+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 shutdown_connections 2024-04-25T11:20:32.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.411+0000 7f6f88cc5640 1 -- 172.21.15.106:0/2025526886 wait complete. 2024-04-25T11:20:32.459 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:32.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1b172f640 1 Processor -- start 2024-04-25T11:20:32.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1b172f640 1 -- start start 2024-04-25T11:20:32.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1b172f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1ac0cecd0 0x7fe1ac0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1b172f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 0x7fe1ac0d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1b172f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1ac0cdb90 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1b172f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe1ac0b30a0 con 0x7fe1ac0cecd0 2024-04-25T11:20:32.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1b172f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ac0b4ee0 con 0x7fe1ac0cf620 2024-04-25T11:20:32.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1aaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1ac0cecd0 0x7fe1ac0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1aaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1ac0cecd0 0x7fe1ac0cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57442/0 (socket says 172.21.15.106:57442) 2024-04-25T11:20:32.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1aaffd640 1 -- 172.21.15.106:0/319929619 learned_addr learned my addr 172.21.15.106:0/319929619 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:32.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1aaffd640 1 -- 172.21.15.106:0/319929619 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 msgr2=0x7fe1ac0d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:32.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1aa7fc640 1 --2- 172.21.15.106:0/319929619 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 0x7fe1ac0d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1aaffd640 1 --2- 172.21.15.106:0/319929619 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 0x7fe1ac0d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1aaffd640 1 -- 172.21.15.106:0/319929619 >> v1:172.21.15.134:6789/0 conn(0x7fe1ac0d07f0 legacy=0x7fe1ac0b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:32.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.603+0000 7fe1aaffd640 1 -- 172.21.15.106:0/319929619 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac0b39a0 con 0x7fe1ac0cecd0 2024-04-25T11:20:32.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1aaffd640 1 --2- 172.21.15.106:0/319929619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0cecd0 0x7fe1ac0cf0e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009870 tx=0x7fe19c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26f54c3ec913ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1a9ffb640 1 -- 172.21.15.106:0/319929619 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c0ab650 con 0x7fe1ac0cecd0 2024-04-25T11:20:32.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1a9ffb640 1 -- 172.21.15.106:0/319929619 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19c0b0070 con 0x7fe1ac0cecd0 2024-04-25T11:20:32.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1a9ffb640 1 -- 172.21.15.106:0/319929619 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c0aba10 con 0x7fe1ac0cecd0 2024-04-25T11:20:32.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1b172f640 1 -- 172.21.15.106:0/319929619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0cecd0 msgr2=0x7fe1ac0cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1b172f640 1 --2- 172.21.15.106:0/319929619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0cecd0 0x7fe1ac0cf0e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009870 tx=0x7fe19c01c4f0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1b172f640 1 -- 172.21.15.106:0/319929619 shutdown_connections 2024-04-25T11:20:32.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1b172f640 1 --2- 172.21.15.106:0/319929619 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 0x7fe1ac0d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1b172f640 1 --2- 172.21.15.106:0/319929619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0cecd0 0x7fe1ac0cf0e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1b172f640 1 -- 172.21.15.106:0/319929619 >> 172.21.15.106:0/319929619 conn(0x7fe1ac0b4020 msgr2=0x7fe1ac0b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1b172f640 1 -- 172.21.15.106:0/319929619 shutdown_connections 2024-04-25T11:20:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.607+0000 7fe1b172f640 1 -- 172.21.15.106:0/319929619 wait complete. 2024-04-25T11:20:32.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 Processor -- start 2024-04-25T11:20:32.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 -- start start 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0cecd0 0x7fe1ac15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 0x7fe1ac09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0d07f0 0x7fe1ac160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac0b4b40 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac0d1010 con 0x7fe1ac0cecd0 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ac0b4cc0 con 0x7fe1ac0cf620 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0d07f0 0x7fe1ac160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0d07f0 0x7fe1ac160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57454/0 (socket says 172.21.15.106:57454) 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 -- 172.21.15.106:0/2766336907 learned_addr learned my addr 172.21.15.106:0/2766336907 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:32.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1aaffd640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0cecd0 0x7fe1ac15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1aa7fc640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 0x7fe1ac09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 -- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 msgr2=0x7fe1ac09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 0x7fe1ac09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 -- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0cecd0 msgr2=0x7fe1ac15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0cecd0 0x7fe1ac15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 -- 172.21.15.106:0/2766336907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac0b1530 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1aaffd640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0cecd0 0x7fe1ac15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1ab7fe640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0d07f0 0x7fe1ac160bb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400b9a0 tx=0x7fe1a400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1937fe640 1 -- 172.21.15.106:0/2766336907 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4019070 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1937fe640 1 -- 172.21.15.106:0/2766336907 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a400cba0 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac16acb0 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1937fe640 1 -- 172.21.15.106:0/2766336907 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4014440 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.611+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac15c210 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.615+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ac16b1d0 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.615+0000 7fe1937fe640 1 -- 172.21.15.106:0/2766336907 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe1a401c070 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.615+0000 7fe1937fe640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe19806f430 0x7fe198071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.615+0000 7fe1937fe640 1 -- 172.21.15.106:0/2766336907 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe1a408ee10 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.615+0000 7fe1aaffd640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe19806f430 0x7fe198071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.615+0000 7fe1aaffd640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe19806f430 0x7fe198071940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0af000 tx=0x7fe19c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.619+0000 7fe1937fe640 1 -- 172.21.15.106:0/2766336907 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe1a405be20 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.779+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1ac1607d0 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.783+0000 7fe1937fe640 1 -- 172.21.15.106:0/2766336907 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16718 (secure 0 0 0) 0x7fe1a405b720 con 0x7fe1ac0d07f0 2024-04-25T11:20:32.784 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":32,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:31.811697+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe19806f430 msgr2=0x7fe198071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe19806f430 0x7fe198071940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0af000 tx=0x7fe19c0a7000 comp rx=0 tx=0).stop 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0d07f0 msgr2=0x7fe1ac160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0d07f0 0x7fe1ac160bb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400b9a0 tx=0x7fe1a400be60 comp rx=0 tx=0).stop 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 shutdown_connections 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe19806f430 0x7fe198071940 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1ac0d07f0 0x7fe1ac160bb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac0cf620 0x7fe1ac09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 --2- 172.21.15.106:0/2766336907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0cecd0 0x7fe1ac15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 >> 172.21.15.106:0/2766336907 conn(0x7fe1ac0b4020 msgr2=0x7fe1ac071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:32.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 shutdown_connections 2024-04-25T11:20:32.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.787+0000 7fe1b172f640 1 -- 172.21.15.106:0/2766336907 wait complete. 2024-04-25T11:20:32.805 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.955+0000 7f932ecc9640 1 Processor -- start 2024-04-25T11:20:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.955+0000 7f932ecc9640 1 -- start start 2024-04-25T11:20:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f932ecc9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280ba970 0x7f93280d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f932ecc9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93280d2160 0x7f93280d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f932ecc9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93280d2ab0 con 0x7f93280b5120 2024-04-25T11:20:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f932ecc9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f93280d1100 con 0x7f93280d2160 2024-04-25T11:20:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f932ecc9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93280d0530 con 0x7f93280ba970 2024-04-25T11:20:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f9327fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93280d2160 0x7f93280d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f9327fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93280d2160 0x7f93280d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57456/0 (socket says 172.21.15.106:57456) 2024-04-25T11:20:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f9327fff640 1 -- 172.21.15.106:0/665938261 learned_addr learned my addr 172.21.15.106:0/665938261 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f932ca65640 1 --2- 172.21.15.106:0/665938261 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280ba970 0x7f93280d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f9327fff640 1 -- 172.21.15.106:0/665938261 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280ba970 msgr2=0x7f93280d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f9327fff640 1 --2- 172.21.15.106:0/665938261 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280ba970 0x7f93280d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f9327fff640 1 -- 172.21.15.106:0/665938261 >> v1:172.21.15.134:6789/0 conn(0x7f93280b5120 legacy=0x7f93280b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.959+0000 7f9327fff640 1 -- 172.21.15.106:0/665938261 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93280b5c40 con 0x7f93280d2160 2024-04-25T11:20:32.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f9327fff640 1 --2- 172.21.15.106:0/665938261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280d2160 0x7f93280d2570 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f931c006fd0 tx=0x7f931c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f9bd10d1f2191f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f93277fe640 1 -- 172.21.15.106:0/665938261 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0b0070 con 0x7f93280d2160 2024-04-25T11:20:32.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f93277fe640 1 -- 172.21.15.106:0/665938261 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c0ab600 con 0x7f93280d2160 2024-04-25T11:20:32.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f93277fe640 1 -- 172.21.15.106:0/665938261 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0ab9d0 con 0x7f93280d2160 2024-04-25T11:20:32.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 -- 172.21.15.106:0/665938261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280d2160 msgr2=0x7f93280d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 --2- 172.21.15.106:0/665938261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280d2160 0x7f93280d2570 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f931c006fd0 tx=0x7f931c01c540 comp rx=0 tx=0).stop 2024-04-25T11:20:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 -- 172.21.15.106:0/665938261 shutdown_connections 2024-04-25T11:20:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 --2- 172.21.15.106:0/665938261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280d2160 0x7f93280d2570 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 --2- 172.21.15.106:0/665938261 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280ba970 0x7f93280d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 -- 172.21.15.106:0/665938261 >> 172.21.15.106:0/665938261 conn(0x7f93280d57e0 msgr2=0x7f93280d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 -- 172.21.15.106:0/665938261 shutdown_connections 2024-04-25T11:20:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 -- 172.21.15.106:0/665938261 wait complete. 2024-04-25T11:20:32.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 Processor -- start 2024-04-25T11:20:32.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.963+0000 7f932ecc9640 1 -- start start 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280b5120 0x7f9328153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280ba970 0x7f932815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d2160 0x7f9328158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93280d1ad0 con 0x7f93280ba970 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93281605b0 con 0x7f93280b5120 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93280b4da0 con 0x7f93280d2160 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280ba970 0x7f932815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280ba970 0x7f932815e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57468/0 (socket says 172.21.15.106:57468) 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 -- 172.21.15.106:0/1093713022 learned_addr learned my addr 172.21.15.106:0/1093713022 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ca65640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280b5120 0x7f9328153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932d266640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d2160 0x7f9328158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 -- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d2160 msgr2=0x7f9328158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d2160 0x7f9328158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 -- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280b5120 msgr2=0x7f9328153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280b5120 0x7f9328153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 -- 172.21.15.106:0/1093713022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9328154400 con 0x7f93280ba970 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ca65640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280b5120 0x7f9328153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f9327fff640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280ba970 0x7f932815e930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f931c01abf0 tx=0x7f931c0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f93257fa640 1 -- 172.21.15.106:0/1093713022 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0b0040 con 0x7f93280ba970 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f93257fa640 1 -- 172.21.15.106:0/1093713022 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c0b9070 con 0x7f93280ba970 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9328159d20 con 0x7f93280ba970 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f93257fa640 1 -- 172.21.15.106:0/1093713022 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0abbf0 con 0x7f93280ba970 2024-04-25T11:20:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9328159760 con 0x7f93280ba970 2024-04-25T11:20:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93280d2ed0 con 0x7f93280ba970 2024-04-25T11:20:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f93257fa640 1 -- 172.21.15.106:0/1093713022 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f931c003620 con 0x7f93280ba970 2024-04-25T11:20:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f93257fa640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f931406f430 0x7f9314071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f93257fa640 1 -- 172.21.15.106:0/1093713022 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f931c128ee0 con 0x7f93280ba970 2024-04-25T11:20:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.967+0000 7f932ca65640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f931406f430 0x7f9314071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.971+0000 7f932ca65640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f931406f430 0x7f9314071940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9318006fd0 tx=0x7f9318008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:32.975+0000 7f93257fa640 1 -- 172.21.15.106:0/1093713022 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f931c016130 con 0x7f93280ba970 2024-04-25T11:20:33.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.131+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f932815d390 con 0x7f93280ba970 2024-04-25T11:20:33.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.131+0000 7f93257fa640 1 -- 172.21.15.106:0/1093713022 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16718 (secure 0 0 0) 0x7f931c0f5dd0 con 0x7f93280ba970 2024-04-25T11:20:33.132 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:33.132 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":32,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:31.811697+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:33.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f931406f430 msgr2=0x7f9314071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f931406f430 0x7f9314071940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9318006fd0 tx=0x7f9318008040 comp rx=0 tx=0).stop 2024-04-25T11:20:33.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280ba970 msgr2=0x7f932815e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280ba970 0x7f932815e930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f931c01abf0 tx=0x7f931c0192e0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 shutdown_connections 2024-04-25T11:20:33.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d2160 0x7f9328158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f931406f430 0x7f9314071940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93280ba970 0x7f932815e930 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 --2- 172.21.15.106:0/1093713022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280b5120 0x7f9328153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 >> 172.21.15.106:0/1093713022 conn(0x7f93280d57e0 msgr2=0x7f93280b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:33.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 shutdown_connections 2024-04-25T11:20:33.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.135+0000 7f932ecc9640 1 -- 172.21.15.106:0/1093713022 wait complete. 2024-04-25T11:20:33.153 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:20:33.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89e1ed7640 1 Processor -- start 2024-04-25T11:20:33.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89e1ed7640 1 -- start start 2024-04-25T11:20:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89e1ed7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f89dc143fa0 0x7f89dc0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89e1ed7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89dc073300 con 0x7f89dc0d1e50 2024-04-25T11:20:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89e1ed7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89dc0735e0 con 0x7f89dc072a20 2024-04-25T11:20:33.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89e1ed7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f89dc0738c0 con 0x7f89dc143fa0 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89dbfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f89dc0d1e50 0x7f89dc0b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34506/0 (socket says 172.21.15.106:34506) 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89dbfff640 1 -- 172.21.15.106:0/3441342125 learned_addr learned my addr 172.21.15.106:0/3441342125 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89db7fe640 1 --2- 172.21.15.106:0/3441342125 >> v2:172.21.15.134:3300/0 conn(0x7f89dc143fa0 0x7f89dc0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121367336 0 0) 0x7f89dc073300 con 0x7f89dc0d1e50 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89c8003530 con 0x7f89dc0d1e50 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1143625665 0 0) 0x7f89c8003530 con 0x7f89dc0d1e50 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 >> v2:172.21.15.134:3300/0 conn(0x7f89dc143fa0 msgr2=0x7f89dc0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 --2- 172.21.15.106:0/3441342125 >> v2:172.21.15.134:3300/0 conn(0x7f89dc143fa0 0x7f89dc0cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 >> v1:172.21.15.134:6789/0 conn(0x7f89dc072a20 legacy=0x7f89dc0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc0b5040 con 0x7f89dc0d1e50 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f89d4001a00 con 0x7f89dc0d1e50 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 >> v1:172.21.15.106:6789/0 conn(0x7f89dc0d1e50 legacy=0x7f89dc0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89c80039c0 0x7f89c8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c8004810 0x7f89c8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c80051d0 0x7f89c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc0738c0 con 0x7f89c8004810 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89c800c170 con 0x7f89c80051d0 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f89c800c370 con 0x7f89c80039c0 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89daffd640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c8004810 0x7f89c8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89db7fe640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89c80039c0 0x7f89c8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89dbfff640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c80051d0 0x7f89c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89daffd640 1 -- 172.21.15.106:0/3441342125 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89c80039c0 msgr2=0x7f89c8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89daffd640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89c80039c0 0x7f89c8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89daffd640 1 -- 172.21.15.106:0/3441342125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c80051d0 msgr2=0x7f89c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89daffd640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c80051d0 0x7f89c800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89daffd640 1 -- 172.21.15.106:0/3441342125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c8003530 con 0x7f89c8004810 2024-04-25T11:20:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.295+0000 7f89dbfff640 1 -- 172.21.15.106:0/3441342125 reap_dead start 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89daffd640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c8004810 0x7f89c8004c20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f89c4001780 tx=0x7f89c401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e440957087278049 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c40013a0 con 0x7f89c8004810 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c40052f0 con 0x7f89c8004810 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89da7fc640 1 -- 172.21.15.106:0/3441342125 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4003840 con 0x7f89c8004810 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3441342125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c8004810 msgr2=0x7f89c8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89e1ed7640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c8004810 0x7f89c8004c20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f89c4001780 tx=0x7f89c401c400 comp rx=0 tx=0).stop 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3441342125 shutdown_connections 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89e1ed7640 1 --2- 172.21.15.106:0/3441342125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c8004810 0x7f89c8004c20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3441342125 >> 172.21.15.106:0/3441342125 conn(0x7f89dc06c9f0 msgr2=0x7f89dc06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3441342125 shutdown_connections 2024-04-25T11:20:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.299+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3441342125 wait complete. 2024-04-25T11:20:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89e1ed7640 1 Processor -- start 2024-04-25T11:20:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89e1ed7640 1 -- start start 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89e1ed7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c8004810 0x7f89dc1326c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89e1ed7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89dc0b2070 0x7f89dc142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89e1ed7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89dc142c70 0x7f89dc15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89e1ed7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc072fb0 con 0x7f89dc142c70 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89e1ed7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc072e30 con 0x7f89c8004810 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89e1ed7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f89dc0cd930 con 0x7f89dc0b2070 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89dc142c70 0x7f89dc15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89dc142c70 0x7f89dc15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57484/0 (socket says 172.21.15.106:57484) 2024-04-25T11:20:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89dbfff640 1 -- 172.21.15.106:0/3886699482 learned_addr learned my addr 172.21.15.106:0/3886699482 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.303+0000 7f89db7fe640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c8004810 0x7f89dc1326c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89daffd640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89dc0b2070 0x7f89dc142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89dbfff640 1 -- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89dc0b2070 msgr2=0x7f89dc142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89dbfff640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89dc0b2070 0x7f89dc142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89dbfff640 1 -- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c8004810 msgr2=0x7f89dc1326c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89dbfff640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c8004810 0x7f89dc1326c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89dbfff640 1 -- 172.21.15.106:0/3886699482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc0b12b0 con 0x7f89dc142c70 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89db7fe640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c8004810 0x7f89dc1326c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89dbfff640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89dc142c70 0x7f89dc15f710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d23e0 tx=0x7f89d4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89d3fff640 1 -- 172.21.15.106:0/3886699482 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d401a070 con 0x7f89dc142c70 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89d3fff640 1 -- 172.21.15.106:0/3886699482 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d4011040 con 0x7f89dc142c70 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89dc0b1590 con 0x7f89dc142c70 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89d3fff640 1 -- 172.21.15.106:0/3886699482 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d400ece0 con 0x7f89dc142c70 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89dc132ef0 con 0x7f89dc142c70 2024-04-25T11:20:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89dc050750 con 0x7f89dc142c70 2024-04-25T11:20:33.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89d3fff640 1 -- 172.21.15.106:0/3886699482 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f89d401d070 con 0x7f89dc142c70 2024-04-25T11:20:33.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89d3fff640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f89bc06f400 0x7f89bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89d3fff640 1 -- 172.21.15.106:0/3886699482 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f89d4091a50 con 0x7f89dc142c70 2024-04-25T11:20:33.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89db7fe640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f89bc06f400 0x7f89bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.307+0000 7f89db7fe640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f89bc06f400 0x7f89bc071910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009aa0 tx=0x7f89cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.315+0000 7f89d3fff640 1 -- 172.21.15.106:0/3886699482 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89d405e9d0 con 0x7f89dc142c70 2024-04-25T11:20:33.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:33.511+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f89dc1601d0 con 0x7f89dc142c70 2024-04-25T11:20:34.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.471+0000 7f89d3fff640 1 -- 172.21.15.106:0/3886699482 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f89d405e2d0 con 0x7f89dc142c70 2024-04-25T11:20:34.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f89bc06f400 msgr2=0x7f89bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f89bc06f400 0x7f89bc071910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009aa0 tx=0x7f89cc008040 comp rx=0 tx=0).stop 2024-04-25T11:20:34.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89dc142c70 msgr2=0x7f89dc15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89dc142c70 0x7f89dc15f710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d23e0 tx=0x7f89d4010d70 comp rx=0 tx=0).stop 2024-04-25T11:20:34.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 shutdown_connections 2024-04-25T11:20:34.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f89bc06f400 0x7f89bc071910 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89dc142c70 0x7f89dc15f710 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89dc0b2070 0x7f89dc142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 --2- 172.21.15.106:0/3886699482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89c8004810 0x7f89dc1326c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 >> 172.21.15.106:0/3886699482 conn(0x7f89dc06c9f0 msgr2=0x7f89dc144b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:34.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 shutdown_connections 2024-04-25T11:20:34.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.475+0000 7f89e1ed7640 1 -- 172.21.15.106:0/3886699482 wait complete. 2024-04-25T11:20:34.499 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:20:34.500 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:20:34.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.655+0000 7fc85cf54640 1 Processor -- start 2024-04-25T11:20:34.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85cf54640 1 -- start start 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85cf54640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8580d1e50 0x7fc85806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85cf54640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc85815d230 0x7fc85815d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85cf54640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8580b68e0 con 0x7fc8580d0cf0 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85cf54640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc858161210 con 0x7fc85815d230 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85cf54640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc8580b6290 con 0x7fc8580d1e50 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc8580d0cf0 0x7fc8580d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34510/0 (socket says 172.21.15.106:34510) 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85659c640 1 -- 172.21.15.106:0/243115420 learned_addr learned my addr 172.21.15.106:0/243115420 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc856d9d640 1 --2- 172.21.15.106:0/243115420 >> v2:172.21.15.134:3300/0 conn(0x7fc85815d230 0x7fc85815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc855d9b640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8580d1e50 0x7fc85806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356892727 0 0) 0x7fc8580b68e0 con 0x7fc8580d0cf0 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc844003530 con 0x7fc8580d0cf0 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2044430768 0 0) 0x7fc844003530 con 0x7fc8580d0cf0 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8580d1e50 msgr2=0x7fc85806ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8580d1e50 0x7fc85806ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 >> v2:172.21.15.134:3300/0 conn(0x7fc85815d230 msgr2=0x7fc85815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 --2- 172.21.15.106:0/243115420 >> v2:172.21.15.134:3300/0 conn(0x7fc85815d230 0x7fc85815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8580b6570 con 0x7fc8580d0cf0 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fc848001a00 con 0x7fc8580d0cf0 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 >> v1:172.21.15.106:6789/0 conn(0x7fc8580d0cf0 legacy=0x7fc8580d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440039c0 0x7fc844003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc844004810 0x7fc844004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc844005160 0x7fc84400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc858161210 con 0x7fc844004810 2024-04-25T11:20:34.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8580b6290 con 0x7fc8440039c0 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc84400bf10 con 0x7fc844005160 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc855d9b640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc844004810 0x7fc844004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85659c640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440039c0 0x7fc844003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc856d9d640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc844005160 0x7fc84400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc855d9b640 1 -- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc844005160 msgr2=0x7fc84400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc855d9b640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc844005160 0x7fc84400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc855d9b640 1 -- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440039c0 msgr2=0x7fc844003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc855d9b640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440039c0 0x7fc844003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc855d9b640 1 -- 172.21.15.106:0/243115420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc844003530 con 0x7fc844004810 2024-04-25T11:20:34.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.659+0000 7fc85659c640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440039c0 0x7fc844003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:34.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc855d9b640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc844004810 0x7fc844004c20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc840009b10 tx=0x7fc84001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29832ce81fe660f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400096b0 con 0x7fc844004810 2024-04-25T11:20:34.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8400a6470 con 0x7fc844004810 2024-04-25T11:20:34.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85559a640 1 -- 172.21.15.106:0/243115420 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400a6860 con 0x7fc844004810 2024-04-25T11:20:34.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85cf54640 1 -- 172.21.15.106:0/243115420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc844004810 msgr2=0x7fc844004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85cf54640 1 --2- 172.21.15.106:0/243115420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc844004810 0x7fc844004c20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc840009b10 tx=0x7fc84001ca90 comp rx=0 tx=0).stop 2024-04-25T11:20:34.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc856d9d640 1 -- 172.21.15.106:0/243115420 reap_dead start 2024-04-25T11:20:34.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85cf54640 1 -- 172.21.15.106:0/243115420 shutdown_connections 2024-04-25T11:20:34.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85cf54640 1 -- 172.21.15.106:0/243115420 >> 172.21.15.106:0/243115420 conn(0x7fc85806c9f0 msgr2=0x7fc85806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:34.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85cf54640 1 -- 172.21.15.106:0/243115420 shutdown_connections 2024-04-25T11:20:34.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.663+0000 7fc85cf54640 1 -- 172.21.15.106:0/243115420 wait complete. 2024-04-25T11:20:34.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 Processor -- start 2024-04-25T11:20:34.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 -- start start 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc844004810 0x7fc8581f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc858159710 0x7fc858159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc858162650 0x7fc858162a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc8580d2060 con 0x7fc858162650 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc85815cdd0 con 0x7fc844004810 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc85815db80 con 0x7fc858159710 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc858162650 0x7fc858162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc858162650 0x7fc858162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57492/0 (socket says 172.21.15.106:57492) 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 -- 172.21.15.106:0/1559178124 learned_addr learned my addr 172.21.15.106:0/1559178124 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85659c640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc844004810 0x7fc8581f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc855d9b640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc858159710 0x7fc858159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 -- 172.21.15.106:0/1559178124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc858159710 msgr2=0x7fc858159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc858159710 0x7fc858159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 -- 172.21.15.106:0/1559178124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc844004810 msgr2=0x7fc8581f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc844004810 0x7fc8581f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 -- 172.21.15.106:0/1559178124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc858162fa0 con 0x7fc858162650 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc856d9d640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc858162650 0x7fc858162a60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc858160430 tx=0x7fc850011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc84effd640 1 -- 172.21.15.106:0/1559178124 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850003620 con 0x7fc858162650 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc84effd640 1 -- 172.21.15.106:0/1559178124 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85001b070 con 0x7fc858162650 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc858155fe0 con 0x7fc858162650 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc84effd640 1 -- 172.21.15.106:0/1559178124 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850003a40 con 0x7fc858162650 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8581565a0 con 0x7fc858162650 2024-04-25T11:20:34.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.667+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc858050750 con 0x7fc858162650 2024-04-25T11:20:34.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.671+0000 7fc84effd640 1 -- 172.21.15.106:0/1559178124 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc850003be0 con 0x7fc858162650 2024-04-25T11:20:34.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.671+0000 7fc84effd640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc83806f400 0x7fc838071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.671+0000 7fc84effd640 1 -- 172.21.15.106:0/1559178124 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc85008f4f0 con 0x7fc858162650 2024-04-25T11:20:34.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.671+0000 7fc85659c640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc83806f400 0x7fc838071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.671+0000 7fc85659c640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc83806f400 0x7fc838071910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc8480014b0 tx=0x7fc848009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.675+0000 7fc84effd640 1 -- 172.21.15.106:0/1559178124 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc850025020 con 0x7fc858162650 2024-04-25T11:20:34.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:34.867+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc8580b45f0 con 0x7fc858162650 2024-04-25T11:20:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.483+0000 7fc84effd640 1 -- 172.21.15.106:0/1559178124 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fc858162fa0 con 0x7fc858162650 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc83806f400 msgr2=0x7fc838071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc83806f400 0x7fc838071910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc8480014b0 tx=0x7fc848009040 comp rx=0 tx=0).stop 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc858162650 msgr2=0x7fc858162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc858162650 0x7fc858162a60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc858160430 tx=0x7fc850011920 comp rx=0 tx=0).stop 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 shutdown_connections 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc83806f400 0x7fc838071910 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc858162650 0x7fc858162a60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc858159710 0x7fc858159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 --2- 172.21.15.106:0/1559178124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc844004810 0x7fc8581f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 >> 172.21.15.106:0/1559178124 conn(0x7fc85806c9f0 msgr2=0x7fc85815b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 shutdown_connections 2024-04-25T11:20:35.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.487+0000 7fc85cf54640 1 -- 172.21.15.106:0/1559178124 wait complete. 2024-04-25T11:20:35.511 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:20:35.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:35.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc529f1c640 1 Processor -- start 2024-04-25T11:20:35.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc529f1c640 1 -- start start 2024-04-25T11:20:35.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc529f1c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 0x7fc5240d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc529f1c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc524163d40 0x7fc524164150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc529f1c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5240d1c20 con 0x7fc52406fc10 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc529f1c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc5241638b0 con 0x7fc524163d40 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc529f1c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc5240b50a0 con 0x7fc5240d12d0 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc5237fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc52406fc10 0x7fc5240b4990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52552/0 (socket says 172.21.15.106:52552) 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc5237fe640 1 -- 172.21.15.106:0/3945517007 learned_addr learned my addr 172.21.15.106:0/3945517007 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc522ffd640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 0x7fc5240d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc523fff640 1 --2- 172.21.15.106:0/3945517007 >> v2:172.21.15.134:3300/0 conn(0x7fc524163d40 0x7fc524164150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 503184539 0 0) 0x7fc514001120 con 0x7fc52406fc10 2024-04-25T11:20:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.663+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc50c003530 con 0x7fc52406fc10 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3640207023 0 0) 0x7fc50c003530 con 0x7fc52406fc10 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 msgr2=0x7fc5240d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 0x7fc5240d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 >> v2:172.21.15.134:3300/0 conn(0x7fc524163d40 msgr2=0x7fc524164150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 --2- 172.21.15.106:0/3945517007 >> v2:172.21.15.134:3300/0 conn(0x7fc524163d40 0x7fc524164150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc524070730 con 0x7fc52406fc10 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fc514001850 con 0x7fc52406fc10 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc522ffd640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 0x7fc5240d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 >> v1:172.21.15.106:6789/0 conn(0x7fc52406fc10 legacy=0x7fc5240b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50c0039c0 0x7fc50c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc50c004810 0x7fc50c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 0x7fc50c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5241638b0 con 0x7fc50c004810 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5240b50a0 con 0x7fc50c0039c0 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c00b9d0 con 0x7fc5240d12d0 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc522ffd640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc50c004810 0x7fc50c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc523fff640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 0x7fc50c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5237fe640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50c0039c0 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc522ffd640 1 -- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 msgr2=0x7fc50c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc522ffd640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 0x7fc50c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc522ffd640 1 -- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50c0039c0 msgr2=0x7fc50c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc522ffd640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50c0039c0 0x7fc50c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc522ffd640 1 -- 172.21.15.106:0/3945517007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c003530 con 0x7fc50c004810 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc523fff640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5240d12d0 0x7fc50c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5237fe640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50c0039c0 0x7fc50c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:35.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc522ffd640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc50c004810 0x7fc50c004c20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc518009c80 tx=0x7fc51801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc85a82da3b599fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:35.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5180b2070 con 0x7fc50c004810 2024-04-25T11:20:35.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5180a57a0 con 0x7fc50c004810 2024-04-25T11:20:35.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc5227fc640 1 -- 172.21.15.106:0/3945517007 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518009880 con 0x7fc50c004810 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc529f1c640 1 -- 172.21.15.106:0/3945517007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc50c004810 msgr2=0x7fc50c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc529f1c640 1 --2- 172.21.15.106:0/3945517007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc50c004810 0x7fc50c004c20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc518009c80 tx=0x7fc51801cb90 comp rx=0 tx=0).stop 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc523fff640 1 -- 172.21.15.106:0/3945517007 reap_dead start 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc529f1c640 1 -- 172.21.15.106:0/3945517007 shutdown_connections 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc529f1c640 1 -- 172.21.15.106:0/3945517007 >> 172.21.15.106:0/3945517007 conn(0x7fc52406c7c0 msgr2=0x7fc52406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc529f1c640 1 -- 172.21.15.106:0/3945517007 shutdown_connections 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.667+0000 7fc529f1c640 1 -- 172.21.15.106:0/3945517007 wait complete. 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 Processor -- start 2024-04-25T11:20:35.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 -- start start 2024-04-25T11:20:35.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50c004810 0x7fc52415cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc524157330 0x7fc5241485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc524147be0 0x7fc52415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc524160db0 con 0x7fc524147be0 2024-04-25T11:20:35.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc524160ad0 con 0x7fc524157330 2024-04-25T11:20:35.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc524161090 con 0x7fc50c004810 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc524147be0 0x7fc52415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc524147be0 0x7fc52415b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60270/0 (socket says 172.21.15.106:60270) 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 -- 172.21.15.106:0/4222249935 learned_addr learned my addr 172.21.15.106:0/4222249935 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc5237fe640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50c004810 0x7fc52415cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc522ffd640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc524157330 0x7fc5241485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 -- 172.21.15.106:0/4222249935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50c004810 msgr2=0x7fc52415cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50c004810 0x7fc52415cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 -- 172.21.15.106:0/4222249935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc524157330 msgr2=0x7fc5241485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc524157330 0x7fc5241485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 -- 172.21.15.106:0/4222249935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52415a040 con 0x7fc524147be0 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc523fff640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc524147be0 0x7fc52415b0e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc5240aec10 tx=0x7fc51c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc50bfff640 1 -- 172.21.15.106:0/4222249935 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c016620 con 0x7fc524147be0 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc50bfff640 1 -- 172.21.15.106:0/4222249935 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51c01b070 con 0x7fc524147be0 2024-04-25T11:20:35.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52415a320 con 0x7fc524147be0 2024-04-25T11:20:35.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc50bfff640 1 -- 172.21.15.106:0/4222249935 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c0169e0 con 0x7fc524147be0 2024-04-25T11:20:35.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.671+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52415a8e0 con 0x7fc524147be0 2024-04-25T11:20:35.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.675+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52406f010 con 0x7fc524147be0 2024-04-25T11:20:35.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.675+0000 7fc50bfff640 1 -- 172.21.15.106:0/4222249935 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc51c016b80 con 0x7fc524147be0 2024-04-25T11:20:35.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.675+0000 7fc50bfff640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc51006f400 0x7fc510071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.675+0000 7fc5237fe640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc51006f400 0x7fc510071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.675+0000 7fc50bfff640 1 -- 172.21.15.106:0/4222249935 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc51c0900d0 con 0x7fc524147be0 2024-04-25T11:20:35.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.675+0000 7fc5237fe640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc51006f400 0x7fc510071910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc514001a00 tx=0x7fc514001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:35.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.679+0000 7fc50bfff640 1 -- 172.21.15.106:0/4222249935 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc51c025020 con 0x7fc524147be0 2024-04-25T11:20:35.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.875+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc52406f670 con 0x7fc524147be0 2024-04-25T11:20:35.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.875+0000 7fc50bfff640 1 -- 172.21.15.106:0/4222249935 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fc51c05d050 con 0x7fc524147be0 2024-04-25T11:20:35.875 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:35.875 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3211345151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3211345151},{"type":"v1","addr":"172.21.15.134:6833","nonce":3211345151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1962020849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1962020849},{"type":"v1","addr":"172.21.15.106:6835","nonce":1962020849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3211705687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3211705687},{"type":"v1","addr":"172.21.15.134:6835","nonce":3211705687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:31.811623+0000","modified":"2024-04-25T11:20:34.872100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1141169456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1141169456},{"type":"v1","addr":"172.21.15.106:6837","nonce":1141169456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T11:20:35.875 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T11:20:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.875+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc51006f400 msgr2=0x7fc510071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.875+0000 7fc529f1c640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc51006f400 0x7fc510071910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc514001a00 tx=0x7fc514001a50 comp rx=0 tx=0).stop 2024-04-25T11:20:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.875+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc524147be0 msgr2=0x7fc52415b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.875+0000 7fc529f1c640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc524147be0 0x7fc52415b0e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc5240aec10 tx=0x7fc51c00f520 comp rx=0 tx=0).stop 2024-04-25T11:20:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.879+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 shutdown_connections 2024-04-25T11:20:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.879+0000 7fc529f1c640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc51006f400 0x7fc510071910 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.879+0000 7fc529f1c640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc524147be0 0x7fc52415b0e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.879+0000 7fc529f1c640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc524157330 0x7fc5241485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.879+0000 7fc529f1c640 1 --2- 172.21.15.106:0/4222249935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50c004810 0x7fc52415cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.879+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 >> 172.21.15.106:0/4222249935 conn(0x7fc52406c7c0 msgr2=0x7fc52415de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.879+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 shutdown_connections 2024-04-25T11:20:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:35.879+0000 7fc529f1c640 1 -- 172.21.15.106:0/4222249935 wait complete. 2024-04-25T11:20:35.896 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:36.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fc04e640 1 Processor -- start 2024-04-25T11:20:36.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fc04e640 1 -- start start 2024-04-25T11:20:36.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fc04e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f51f415e0a0 0x7f51f4073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fc04e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f40d0ce0 0x7f51f40d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fc04e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f40ce050 con 0x7f51f40ce850 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fc04e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f51f415e6b0 con 0x7f51f415e0a0 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fc04e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51f415e990 con 0x7f51f40d0ce0 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f95e9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f51f40ce850 0x7f51f40cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52558/0 (socket says 172.21.15.106:52558) 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f95e9640 1 -- 172.21.15.106:0/1593539200 learned_addr learned my addr 172.21.15.106:0/1593539200 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f9dea640 1 --2- 172.21.15.106:0/1593539200 >> v2:172.21.15.134:3300/0 conn(0x7f51f415e0a0 0x7f51f4073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fa5eb640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f40d0ce0 0x7f51f40d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736299817 0 0) 0x7f51f40ce050 con 0x7f51f40ce850 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51e0003530 con 0x7f51f40ce850 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4142402523 0 0) 0x7f51e0003530 con 0x7f51f40ce850 2024-04-25T11:20:36.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f40d0ce0 msgr2=0x7f51f40d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f40d0ce0 0x7f51f40d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 >> v2:172.21.15.134:3300/0 conn(0x7f51f415e0a0 msgr2=0x7f51f4073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 --2- 172.21.15.106:0/1593539200 >> v2:172.21.15.134:3300/0 conn(0x7f51f415e0a0 0x7f51f4073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f415ec70 con 0x7f51f40ce850 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f51f0001680 con 0x7f51f40ce850 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 >> v1:172.21.15.106:6789/0 conn(0x7f51f40ce850 legacy=0x7f51f40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f40d0ce0 0x7f51e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51e0003f40 0x7f51e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51e0004d50 0x7f51e0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f51f415e6b0 con 0x7f51e0003f40 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51f415e990 con 0x7f51e0004d50 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51e000ba20 con 0x7f51f40d0ce0 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f95e9640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51e0003f40 0x7f51e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f9dea640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f40d0ce0 0x7f51e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51fa5eb640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51e0004d50 0x7f51e0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f95e9640 1 -- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f40d0ce0 msgr2=0x7f51e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f95e9640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f40d0ce0 0x7f51e00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f95e9640 1 -- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51e0004d50 msgr2=0x7f51e0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f95e9640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51e0004d50 0x7f51e0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.031+0000 7f51f95e9640 1 -- 172.21.15.106:0/1593539200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e0003530 con 0x7f51e0003f40 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51f95e9640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51e0003f40 0x7f51e0004810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f51f0001060 tx=0x7f51f001f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bfa00f424fb029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0007d60 con 0x7f51e0003f40 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0005ae0 con 0x7f51e0003f40 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51f8de8640 1 -- 172.21.15.106:0/1593539200 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0007070 con 0x7f51e0003f40 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51fc04e640 1 -- 172.21.15.106:0/1593539200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51e0003f40 msgr2=0x7f51e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51fc04e640 1 --2- 172.21.15.106:0/1593539200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51e0003f40 0x7f51e0004810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f51f0001060 tx=0x7f51f001f3d0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51fa5eb640 1 -- 172.21.15.106:0/1593539200 reap_dead start 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51fc04e640 1 -- 172.21.15.106:0/1593539200 shutdown_connections 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51fc04e640 1 -- 172.21.15.106:0/1593539200 >> 172.21.15.106:0/1593539200 conn(0x7f51f406c7c0 msgr2=0x7f51f406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51fc04e640 1 -- 172.21.15.106:0/1593539200 shutdown_connections 2024-04-25T11:20:36.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.035+0000 7f51fc04e640 1 -- 172.21.15.106:0/1593539200 wait complete. 2024-04-25T11:20:36.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 Processor -- start 2024-04-25T11:20:36.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 -- start start 2024-04-25T11:20:36.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f41e42b0 0x7f51f41f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51f41e46c0 0x7f51f413dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f413e1e0 0x7f51f41f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f51f40d0860 con 0x7f51f41e46c0 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51f40d06e0 con 0x7f51f413e1e0 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51f415f270 con 0x7f51f41e42b0 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51f41e46c0 0x7f51f413dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51f41e46c0 0x7f51f413dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60280/0 (socket says 172.21.15.106:60280) 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 -- 172.21.15.106:0/242200855 learned_addr learned my addr 172.21.15.106:0/242200855 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f9dea640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f41e42b0 0x7f51f41f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fa5eb640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f413e1e0 0x7f51f41f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 -- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f41e42b0 msgr2=0x7f51f41f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f41e42b0 0x7f51f41f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 -- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f413e1e0 msgr2=0x7f51f41f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f413e1e0 0x7f51f41f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 -- 172.21.15.106:0/242200855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f413e5f0 con 0x7f51f41e46c0 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f9dea640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f41e42b0 0x7f51f41f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51f95e9640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51f41e46c0 0x7f51f413dca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f51f001fc70 tx=0x7f51e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51ea7fc640 1 -- 172.21.15.106:0/242200855 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00048b0 con 0x7f51f41e46c0 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51ea7fc640 1 -- 172.21.15.106:0/242200855 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0004a50 con 0x7f51f41e46c0 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f41e4c20 con 0x7f51f41e46c0 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51ea7fc640 1 -- 172.21.15.106:0/242200855 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0004e00 con 0x7f51f41e46c0 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f41e51e0 con 0x7f51f41e46c0 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.039+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f406ce20 con 0x7f51f41e46c0 2024-04-25T11:20:36.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.043+0000 7f51ea7fc640 1 -- 172.21.15.106:0/242200855 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f51f0007070 con 0x7f51f41e46c0 2024-04-25T11:20:36.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.043+0000 7f51ea7fc640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f51d806f400 0x7f51d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.043+0000 7f51ea7fc640 1 -- 172.21.15.106:0/242200855 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f51f012c2c0 con 0x7f51f41e46c0 2024-04-25T11:20:36.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.043+0000 7f51f9dea640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f51d806f400 0x7f51d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.043+0000 7f51f9dea640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f51d806f400 0x7f51d8071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f51f41f71c0 tx=0x7f51e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.047+0000 7f51ea7fc640 1 -- 172.21.15.106:0/242200855 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f51f413e5f0 con 0x7f51f41e46c0 2024-04-25T11:20:36.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.199+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51f406da30 con 0x7f51f41e46c0 2024-04-25T11:20:36.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.199+0000 7f51ea7fc640 1 -- 172.21.15.106:0/242200855 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7f51f00190e0 con 0x7f51f41e46c0 2024-04-25T11:20:36.200 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:36.200 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:33.406557+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f51d806f400 msgr2=0x7f51d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f51d806f400 0x7f51d8071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f51f41f71c0 tx=0x7f51e4008040 comp rx=0 tx=0).stop 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51f41e46c0 msgr2=0x7f51f413dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51f41e46c0 0x7f51f413dca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f51f001fc70 tx=0x7f51e0005840 comp rx=0 tx=0).stop 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 shutdown_connections 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f413e1e0 0x7f51f41f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f51d806f400 0x7f51d8071910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51f41e46c0 0x7f51f413dca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 --2- 172.21.15.106:0/242200855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51f41e42b0 0x7f51f41f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 >> 172.21.15.106:0/242200855 conn(0x7f51f406c7c0 msgr2=0x7f51f4142b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 shutdown_connections 2024-04-25T11:20:36.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.203+0000 7f51fc04e640 1 -- 172.21.15.106:0/242200855 wait complete. 2024-04-25T11:20:36.220 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:20:36.220 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:20:36.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.367+0000 7f4506f7d640 1 Processor -- start 2024-04-25T11:20:36.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.367+0000 7f4506f7d640 1 -- start start 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4506f7d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4500143e60 0x7f45000cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4506f7d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f45000ce7c0 0x7f45000cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4506f7d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45000cf110 con 0x7f4500072a50 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4506f7d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f45000d06e0 con 0x7f4500143e60 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4506f7d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4500144270 con 0x7f45000ce7c0 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4504d19640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4500143e60 0x7f45000cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4504d19640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4500143e60 0x7f45000cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60286/0 (socket says 172.21.15.106:60286) 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4504d19640 1 -- 172.21.15.106:0/2340499362 learned_addr learned my addr 172.21.15.106:0/2340499362 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f44fffff640 1 --2- 172.21.15.106:0/2340499362 >> v2:172.21.15.134:3300/0 conn(0x7f45000ce7c0 0x7f45000cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4504d19640 1 -- 172.21.15.106:0/2340499362 >> v2:172.21.15.134:3300/0 conn(0x7f45000ce7c0 msgr2=0x7f45000cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4504d19640 1 --2- 172.21.15.106:0/2340499362 >> v2:172.21.15.134:3300/0 conn(0x7f45000ce7c0 0x7f45000cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4504d19640 1 -- 172.21.15.106:0/2340499362 >> v1:172.21.15.134:6789/0 conn(0x7f4500072a50 legacy=0x7f4500072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.371+0000 7f4504d19640 1 -- 172.21.15.106:0/2340499362 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45000d0980 con 0x7f4500143e60 2024-04-25T11:20:36.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4504d19640 1 --2- 172.21.15.106:0/2340499362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500143e60 0x7f45000cdd90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f45000b6700 tx=0x7f44f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bade0c8fa6bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f44ff7fe640 1 -- 172.21.15.106:0/2340499362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40aa650 con 0x7f4500143e60 2024-04-25T11:20:36.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f44ff7fe640 1 -- 172.21.15.106:0/2340499362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f40af070 con 0x7f4500143e60 2024-04-25T11:20:36.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f44ff7fe640 1 -- 172.21.15.106:0/2340499362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40aaa10 con 0x7f4500143e60 2024-04-25T11:20:36.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 -- 172.21.15.106:0/2340499362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500143e60 msgr2=0x7f45000cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 --2- 172.21.15.106:0/2340499362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500143e60 0x7f45000cdd90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f45000b6700 tx=0x7f44f401c4f0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 -- 172.21.15.106:0/2340499362 shutdown_connections 2024-04-25T11:20:36.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 --2- 172.21.15.106:0/2340499362 >> v2:172.21.15.134:3300/0 conn(0x7f45000ce7c0 0x7f45000cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 --2- 172.21.15.106:0/2340499362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500143e60 0x7f45000cdd90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 -- 172.21.15.106:0/2340499362 >> 172.21.15.106:0/2340499362 conn(0x7f450006c7c0 msgr2=0x7f450006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 -- 172.21.15.106:0/2340499362 shutdown_connections 2024-04-25T11:20:36.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 -- 172.21.15.106:0/2340499362 wait complete. 2024-04-25T11:20:36.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 Processor -- start 2024-04-25T11:20:36.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.375+0000 7f4506f7d640 1 -- start start 2024-04-25T11:20:36.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500072a50 0x7f450015cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ce7c0 0x7f45001458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4500143e60 0x7f45001604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45000ce2d0 con 0x7f4500072a50 2024-04-25T11:20:36.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45000d1500 con 0x7f4500143e60 2024-04-25T11:20:36.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45000ce450 con 0x7f45000ce7c0 2024-04-25T11:20:36.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500072a50 0x7f450015cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500072a50 0x7f450015cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60302/0 (socket says 172.21.15.106:60302) 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 -- 172.21.15.106:0/546456729 learned_addr learned my addr 172.21.15.106:0/546456729 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fffff640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ce7c0 0x7f45001458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f450551a640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4500143e60 0x7f45001604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 -- 172.21.15.106:0/546456729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ce7c0 msgr2=0x7f45001458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ce7c0 0x7f45001458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 -- 172.21.15.106:0/546456729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4500143e60 msgr2=0x7f45001604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4500143e60 0x7f45001604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 -- 172.21.15.106:0/546456729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450015c1c0 con 0x7f4500072a50 2024-04-25T11:20:36.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4504d19640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500072a50 0x7f450015cf80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f44f4019100 tx=0x7f44f401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.379 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f872eeb7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fd7fa640 1 -- 172.21.15.106:0/546456729 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40bd5f0 con 0x7f4500072a50 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fd7fa640 1 -- 172.21.15.106:0/546456729 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f40a6040 con 0x7f4500072a50 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fd7fa640 1 -- 172.21.15.106:0/546456729 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40bd9d0 con 0x7f4500072a50 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450015c4a0 con 0x7f4500072a50 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450015ca60 con 0x7f4500072a50 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4500143040 con 0x7f4500072a50 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fd7fa640 1 -- 172.21.15.106:0/546456729 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f44f40b33f0 con 0x7f4500072a50 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fd7fa640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f44e406f400 0x7f44e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fffff640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f44e406f400 0x7f44e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fd7fa640 1 -- 172.21.15.106:0/546456729 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f44f4129870 con 0x7f4500072a50 2024-04-25T11:20:36.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.379+0000 7f44fffff640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f44e406f400 0x7f44e4071910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f450013a0f0 tx=0x7f44ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.387+0000 7f44fd7fa640 1 -- 172.21.15.106:0/546456729 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44f40c3060 con 0x7f4500072a50 2024-04-25T11:20:36.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.551+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4500142d60 con 0x7f44e406f400 2024-04-25T11:20:36.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.559+0000 7f44fd7fa640 1 -- 172.21.15.106:0/546456729 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251901 (secure 0 0 0) 0x7f4500142d60 con 0x7f44e406f400 2024-04-25T11:20:36.562 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:36.564 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f44e406f400 msgr2=0x7f44e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f44e406f400 0x7f44e4071910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f450013a0f0 tx=0x7f44ec008040 comp rx=0 tx=0).stop 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500072a50 msgr2=0x7f450015cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500072a50 0x7f450015cf80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f44f4019100 tx=0x7f44f401ad30 comp rx=0 tx=0).stop 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 shutdown_connections 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4500143e60 0x7f45001604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ce7c0 0x7f45001458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f44e406f400 0x7f44e4071910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 --2- 172.21.15.106:0/546456729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4500072a50 0x7f450015cf80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 >> 172.21.15.106:0/546456729 conn(0x7f450006c7c0 msgr2=0x7f4500143560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 shutdown_connections 2024-04-25T11:20:36.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.563+0000 7f4506f7d640 1 -- 172.21.15.106:0/546456729 wait complete. 2024-04-25T11:20:36.582 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T11:20:36.113546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220232,"kb_used_data":4064,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754488,"statfs":{"total":773094113280,"available":772868595712,"internally_reserved":0,"allocated":4161536,"data_stored":2320620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.720323"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.474260+0000","last_change":"2024-04-25T11:20:32.465790+0000","last_active":"2024-04-25T11:20:32.474260+0000","last_peered":"2024-04-25T11:20:32.474260+0000","last_clean":"2024-04-25T11:20:32.474260+0000","last_became_active":"2024-04-25T11:20:32.459298+0000","last_became_peered":"2024-04-25T11:20:32.459298+0000","last_unstale":"2024-04-25T11:20:32.474260+0000","last_undegraded":"2024-04-25T11:20:32.474260+0000","last_fullsized":"2024-04-25T11:20:32.474260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:03.133009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902433+0000","last_change":"2024-04-25T11:20:32.428769+0000","last_active":"2024-04-25T11:20:33.902433+0000","last_peered":"2024-04-25T11:20:33.902433+0000","last_clean":"2024-04-25T11:20:33.902433+0000","last_became_active":"2024-04-25T11:20:32.428420+0000","last_became_peered":"2024-04-25T11:20:32.428420+0000","last_unstale":"2024-04-25T11:20:33.902433+0000","last_undegraded":"2024-04-25T11:20:33.902433+0000","last_fullsized":"2024-04-25T11:20:33.902433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:47.880494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.478949+0000","last_change":"2024-04-25T11:20:32.456294+0000","last_active":"2024-04-25T11:20:32.478949+0000","last_peered":"2024-04-25T11:20:32.478949+0000","last_clean":"2024-04-25T11:20:32.478949+0000","last_became_active":"2024-04-25T11:20:32.455501+0000","last_became_peered":"2024-04-25T11:20:32.455501+0000","last_unstale":"2024-04-25T11:20:32.478949+0000","last_undegraded":"2024-04-25T11:20:32.478949+0000","last_fullsized":"2024-04-25T11:20:32.478949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:56.022841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.441235+0000","last_change":"2024-04-25T11:20:32.440116+0000","last_active":"2024-04-25T11:20:32.441235+0000","last_peered":"2024-04-25T11:20:32.441235+0000","last_clean":"2024-04-25T11:20:32.441235+0000","last_became_active":"2024-04-25T11:20:32.439588+0000","last_became_peered":"2024-04-25T11:20:32.439588+0000","last_unstale":"2024-04-25T11:20:32.441235+0000","last_undegraded":"2024-04-25T11:20:32.441235+0000","last_fullsized":"2024-04-25T11:20:32.441235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415470+0000","last_change":"2024-04-25T11:20:32.438053+0000","last_active":"2024-04-25T11:20:33.415470+0000","last_peered":"2024-04-25T11:20:33.415470+0000","last_clean":"2024-04-25T11:20:33.415470+0000","last_became_active":"2024-04-25T11:20:32.437359+0000","last_became_peered":"2024-04-25T11:20:32.437359+0000","last_unstale":"2024-04-25T11:20:33.415470+0000","last_undegraded":"2024-04-25T11:20:33.415470+0000","last_fullsized":"2024-04-25T11:20:33.415470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:19.148902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901892+0000","last_change":"2024-04-25T11:20:32.434766+0000","last_active":"2024-04-25T11:20:33.901892+0000","last_peered":"2024-04-25T11:20:33.901892+0000","last_clean":"2024-04-25T11:20:33.901892+0000","last_became_active":"2024-04-25T11:20:32.434350+0000","last_became_peered":"2024-04-25T11:20:32.434350+0000","last_unstale":"2024-04-25T11:20:33.901892+0000","last_undegraded":"2024-04-25T11:20:33.901892+0000","last_fullsized":"2024-04-25T11:20:33.901892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:56.806426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.470968+0000","last_change":"2024-04-25T11:20:32.469167+0000","last_active":"2024-04-25T11:20:32.470968+0000","last_peered":"2024-04-25T11:20:32.470968+0000","last_clean":"2024-04-25T11:20:32.470968+0000","last_became_active":"2024-04-25T11:20:32.468381+0000","last_became_peered":"2024-04-25T11:20:32.468381+0000","last_unstale":"2024-04-25T11:20:32.470968+0000","last_undegraded":"2024-04-25T11:20:32.470968+0000","last_fullsized":"2024-04-25T11:20:32.470968+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:11.840736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.471363+0000","last_change":"2024-04-25T11:20:32.462963+0000","last_active":"2024-04-25T11:20:32.471363+0000","last_peered":"2024-04-25T11:20:32.471363+0000","last_clean":"2024-04-25T11:20:32.471363+0000","last_became_active":"2024-04-25T11:20:32.462132+0000","last_became_peered":"2024-04-25T11:20:32.462132+0000","last_unstale":"2024-04-25T11:20:32.471363+0000","last_undegraded":"2024-04-25T11:20:32.471363+0000","last_fullsized":"2024-04-25T11:20:32.471363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:03.896712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.434863+0000","last_change":"2024-04-25T11:20:32.430790+0000","last_active":"2024-04-25T11:20:32.434863+0000","last_peered":"2024-04-25T11:20:32.434863+0000","last_clean":"2024-04-25T11:20:32.434863+0000","last_became_active":"2024-04-25T11:20:32.429907+0000","last_became_peered":"2024-04-25T11:20:32.429907+0000","last_unstale":"2024-04-25T11:20:32.434863+0000","last_undegraded":"2024-04-25T11:20:32.434863+0000","last_fullsized":"2024-04-25T11:20:32.434863+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:22.158494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.437926+0000","last_change":"2024-04-25T11:20:32.433782+0000","last_active":"2024-04-25T11:20:32.437926+0000","last_peered":"2024-04-25T11:20:32.437926+0000","last_clean":"2024-04-25T11:20:32.437926+0000","last_became_active":"2024-04-25T11:20:32.433477+0000","last_became_peered":"2024-04-25T11:20:32.433477+0000","last_unstale":"2024-04-25T11:20:32.437926+0000","last_undegraded":"2024-04-25T11:20:32.437926+0000","last_fullsized":"2024-04-25T11:20:32.437926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:21.990942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.470073+0000","last_change":"2024-04-25T11:20:32.454287+0000","last_active":"2024-04-25T11:20:32.470073+0000","last_peered":"2024-04-25T11:20:32.470073+0000","last_clean":"2024-04-25T11:20:32.470073+0000","last_became_active":"2024-04-25T11:20:32.453370+0000","last_became_peered":"2024-04-25T11:20:32.453370+0000","last_unstale":"2024-04-25T11:20:32.470073+0000","last_undegraded":"2024-04-25T11:20:32.470073+0000","last_fullsized":"2024-04-25T11:20:32.470073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:54.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479018+0000","last_change":"2024-04-25T11:20:32.454554+0000","last_active":"2024-04-25T11:20:32.479018+0000","last_peered":"2024-04-25T11:20:32.479018+0000","last_clean":"2024-04-25T11:20:32.479018+0000","last_became_active":"2024-04-25T11:20:32.447829+0000","last_became_peered":"2024-04-25T11:20:32.447829+0000","last_unstale":"2024-04-25T11:20:32.479018+0000","last_undegraded":"2024-04-25T11:20:32.479018+0000","last_fullsized":"2024-04-25T11:20:32.479018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:29.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415752+0000","last_change":"2024-04-25T11:20:32.430914+0000","last_active":"2024-04-25T11:20:33.415752+0000","last_peered":"2024-04-25T11:20:33.415752+0000","last_clean":"2024-04-25T11:20:33.415752+0000","last_became_active":"2024-04-25T11:20:32.430218+0000","last_became_peered":"2024-04-25T11:20:32.430218+0000","last_unstale":"2024-04-25T11:20:33.415752+0000","last_undegraded":"2024-04-25T11:20:33.415752+0000","last_fullsized":"2024-04-25T11:20:33.415752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:51.320902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902359+0000","last_change":"2024-04-25T11:20:32.429534+0000","last_active":"2024-04-25T11:20:33.902359+0000","last_peered":"2024-04-25T11:20:33.902359+0000","last_clean":"2024-04-25T11:20:33.902359+0000","last_became_active":"2024-04-25T11:20:32.428338+0000","last_became_peered":"2024-04-25T11:20:32.428338+0000","last_unstale":"2024-04-25T11:20:33.902359+0000","last_undegraded":"2024-04-25T11:20:33.902359+0000","last_fullsized":"2024-04-25T11:20:33.902359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:09.836157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.482268+0000","last_change":"2024-04-25T11:20:32.472104+0000","last_active":"2024-04-25T11:20:32.482268+0000","last_peered":"2024-04-25T11:20:32.482268+0000","last_clean":"2024-04-25T11:20:32.482268+0000","last_became_active":"2024-04-25T11:20:32.470774+0000","last_became_peered":"2024-04-25T11:20:32.470774+0000","last_unstale":"2024-04-25T11:20:32.482268+0000","last_undegraded":"2024-04-25T11:20:32.482268+0000","last_fullsized":"2024-04-25T11:20:32.482268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:44.004137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.475658+0000","last_change":"2024-04-25T11:20:32.471795+0000","last_active":"2024-04-25T11:20:32.475658+0000","last_peered":"2024-04-25T11:20:32.475658+0000","last_clean":"2024-04-25T11:20:32.475658+0000","last_became_active":"2024-04-25T11:20:32.470430+0000","last_became_peered":"2024-04-25T11:20:32.470430+0000","last_unstale":"2024-04-25T11:20:32.475658+0000","last_undegraded":"2024-04-25T11:20:32.475658+0000","last_fullsized":"2024-04-25T11:20:32.475658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.006165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.480439+0000","last_change":"2024-04-25T11:20:32.476501+0000","last_active":"2024-04-25T11:20:32.480439+0000","last_peered":"2024-04-25T11:20:32.480439+0000","last_clean":"2024-04-25T11:20:32.480439+0000","last_became_active":"2024-04-25T11:20:32.475810+0000","last_became_peered":"2024-04-25T11:20:32.475810+0000","last_unstale":"2024-04-25T11:20:32.480439+0000","last_undegraded":"2024-04-25T11:20:32.480439+0000","last_fullsized":"2024-04-25T11:20:32.480439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:44.004137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902484+0000","last_change":"2024-04-25T11:20:32.436111+0000","last_active":"2024-04-25T11:20:33.902484+0000","last_peered":"2024-04-25T11:20:33.902484+0000","last_clean":"2024-04-25T11:20:33.902484+0000","last_became_active":"2024-04-25T11:20:32.434315+0000","last_became_peered":"2024-04-25T11:20:32.434315+0000","last_unstale":"2024-04-25T11:20:33.902484+0000","last_undegraded":"2024-04-25T11:20:33.902484+0000","last_fullsized":"2024-04-25T11:20:33.902484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.780852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902140+0000","last_change":"2024-04-25T11:20:32.426964+0000","last_active":"2024-04-25T11:20:33.902140+0000","last_peered":"2024-04-25T11:20:33.902140+0000","last_clean":"2024-04-25T11:20:33.902140+0000","last_became_active":"2024-04-25T11:20:32.426586+0000","last_became_peered":"2024-04-25T11:20:32.426586+0000","last_unstale":"2024-04-25T11:20:33.902140+0000","last_undegraded":"2024-04-25T11:20:33.902140+0000","last_fullsized":"2024-04-25T11:20:33.902140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:15.990703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415952+0000","last_change":"2024-04-25T11:20:32.463304+0000","last_active":"2024-04-25T11:20:33.415952+0000","last_peered":"2024-04-25T11:20:33.415952+0000","last_clean":"2024-04-25T11:20:33.415952+0000","last_became_active":"2024-04-25T11:20:32.463001+0000","last_became_peered":"2024-04-25T11:20:32.463001+0000","last_unstale":"2024-04-25T11:20:33.415952+0000","last_undegraded":"2024-04-25T11:20:33.415952+0000","last_fullsized":"2024-04-25T11:20:33.415952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:03.133009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.444633+0000","last_change":"2024-04-25T11:20:32.434745+0000","last_active":"2024-04-25T11:20:32.444633+0000","last_peered":"2024-04-25T11:20:32.444633+0000","last_clean":"2024-04-25T11:20:32.444633+0000","last_became_active":"2024-04-25T11:20:32.429708+0000","last_became_peered":"2024-04-25T11:20:32.429708+0000","last_unstale":"2024-04-25T11:20:32.444633+0000","last_undegraded":"2024-04-25T11:20:32.444633+0000","last_fullsized":"2024-04-25T11:20:32.444633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:54.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.434282+0000","last_change":"2024-04-25T11:20:32.424086+0000","last_active":"2024-04-25T11:20:32.434282+0000","last_peered":"2024-04-25T11:20:32.434282+0000","last_clean":"2024-04-25T11:20:32.434282+0000","last_became_active":"2024-04-25T11:20:32.423314+0000","last_became_peered":"2024-04-25T11:20:32.423314+0000","last_unstale":"2024-04-25T11:20:32.434282+0000","last_undegraded":"2024-04-25T11:20:32.434282+0000","last_fullsized":"2024-04-25T11:20:32.434282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.414723+0000","last_change":"2024-04-25T11:20:32.449227+0000","last_active":"2024-04-25T11:20:33.414723+0000","last_peered":"2024-04-25T11:20:33.414723+0000","last_clean":"2024-04-25T11:20:33.414723+0000","last_became_active":"2024-04-25T11:20:32.448875+0000","last_became_peered":"2024-04-25T11:20:32.448875+0000","last_unstale":"2024-04-25T11:20:33.414723+0000","last_undegraded":"2024-04-25T11:20:33.414723+0000","last_fullsized":"2024-04-25T11:20:33.414723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.006165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.481897+0000","last_change":"2024-04-25T11:20:32.462363+0000","last_active":"2024-04-25T11:20:32.481897+0000","last_peered":"2024-04-25T11:20:32.481897+0000","last_clean":"2024-04-25T11:20:32.481897+0000","last_became_active":"2024-04-25T11:20:32.461569+0000","last_became_peered":"2024-04-25T11:20:32.461569+0000","last_unstale":"2024-04-25T11:20:32.481897+0000","last_undegraded":"2024-04-25T11:20:32.481897+0000","last_fullsized":"2024-04-25T11:20:32.481897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:34.139129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.430637+0000","last_change":"2024-04-25T11:20:32.428856+0000","last_active":"2024-04-25T11:20:32.430637+0000","last_peered":"2024-04-25T11:20:32.430637+0000","last_clean":"2024-04-25T11:20:32.430637+0000","last_became_active":"2024-04-25T11:20:32.428384+0000","last_became_peered":"2024-04-25T11:20:32.428384+0000","last_unstale":"2024-04-25T11:20:32.430637+0000","last_undegraded":"2024-04-25T11:20:32.430637+0000","last_fullsized":"2024-04-25T11:20:32.430637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902890+0000","last_change":"2024-04-25T11:20:32.450266+0000","last_active":"2024-04-25T11:20:33.902890+0000","last_peered":"2024-04-25T11:20:33.902890+0000","last_clean":"2024-04-25T11:20:33.902890+0000","last_became_active":"2024-04-25T11:20:32.449936+0000","last_became_peered":"2024-04-25T11:20:32.449936+0000","last_unstale":"2024-04-25T11:20:33.902890+0000","last_undegraded":"2024-04-25T11:20:33.902890+0000","last_fullsized":"2024-04-25T11:20:33.902890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:43.455079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.438271+0000","last_change":"2024-04-25T11:20:32.436352+0000","last_active":"2024-04-25T11:20:32.438271+0000","last_peered":"2024-04-25T11:20:32.438271+0000","last_clean":"2024-04-25T11:20:32.438271+0000","last_became_active":"2024-04-25T11:20:32.436034+0000","last_became_peered":"2024-04-25T11:20:32.436034+0000","last_unstale":"2024-04-25T11:20:32.438271+0000","last_undegraded":"2024-04-25T11:20:32.438271+0000","last_fullsized":"2024-04-25T11:20:32.438271+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:54.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479062+0000","last_change":"2024-04-25T11:20:32.455350+0000","last_active":"2024-04-25T11:20:32.479062+0000","last_peered":"2024-04-25T11:20:32.479062+0000","last_clean":"2024-04-25T11:20:32.479062+0000","last_became_active":"2024-04-25T11:20:32.454456+0000","last_became_peered":"2024-04-25T11:20:32.454456+0000","last_unstale":"2024-04-25T11:20:32.479062+0000","last_undegraded":"2024-04-25T11:20:32.479062+0000","last_fullsized":"2024-04-25T11:20:32.479062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:11.840736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.432238+0000","last_change":"2024-04-25T11:20:32.423941+0000","last_active":"2024-04-25T11:20:32.432238+0000","last_peered":"2024-04-25T11:20:32.432238+0000","last_clean":"2024-04-25T11:20:32.432238+0000","last_became_active":"2024-04-25T11:20:32.423154+0000","last_became_peered":"2024-04-25T11:20:32.423154+0000","last_unstale":"2024-04-25T11:20:32.432238+0000","last_undegraded":"2024-04-25T11:20:32.432238+0000","last_fullsized":"2024-04-25T11:20:32.432238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:18.525729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.470230+0000","last_change":"2024-04-25T11:20:32.463065+0000","last_active":"2024-04-25T11:20:32.470230+0000","last_peered":"2024-04-25T11:20:32.470230+0000","last_clean":"2024-04-25T11:20:32.470230+0000","last_became_active":"2024-04-25T11:20:32.462335+0000","last_became_peered":"2024-04-25T11:20:32.462335+0000","last_unstale":"2024-04-25T11:20:32.470230+0000","last_undegraded":"2024-04-25T11:20:32.470230+0000","last_fullsized":"2024-04-25T11:20:32.470230+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.473547+0000","last_change":"2024-04-25T11:20:32.466349+0000","last_active":"2024-04-25T11:20:32.473547+0000","last_peered":"2024-04-25T11:20:32.473547+0000","last_clean":"2024-04-25T11:20:32.473547+0000","last_became_active":"2024-04-25T11:20:32.465576+0000","last_became_peered":"2024-04-25T11:20:32.465576+0000","last_unstale":"2024-04-25T11:20:32.473547+0000","last_undegraded":"2024-04-25T11:20:32.473547+0000","last_fullsized":"2024-04-25T11:20:32.473547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:30.601537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479486+0000","last_change":"2024-04-25T11:20:32.473437+0000","last_active":"2024-04-25T11:20:32.479486+0000","last_peered":"2024-04-25T11:20:32.479486+0000","last_clean":"2024-04-25T11:20:32.479486+0000","last_became_active":"2024-04-25T11:20:32.472793+0000","last_became_peered":"2024-04-25T11:20:32.472793+0000","last_unstale":"2024-04-25T11:20:32.479486+0000","last_undegraded":"2024-04-25T11:20:32.479486+0000","last_fullsized":"2024-04-25T11:20:32.479486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:56.022841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413561+0000","last_change":"2024-04-25T11:20:32.428037+0000","last_active":"2024-04-25T11:20:33.413561+0000","last_peered":"2024-04-25T11:20:33.413561+0000","last_clean":"2024-04-25T11:20:33.413561+0000","last_became_active":"2024-04-25T11:20:32.427628+0000","last_became_peered":"2024-04-25T11:20:32.427628+0000","last_unstale":"2024-04-25T11:20:33.413561+0000","last_undegraded":"2024-04-25T11:20:33.413561+0000","last_fullsized":"2024-04-25T11:20:33.413561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:29.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.482272+0000","last_change":"2024-04-25T11:20:31.455421+0000","last_active":"2024-04-25T11:20:32.482272+0000","last_peered":"2024-04-25T11:20:32.482272+0000","last_clean":"2024-04-25T11:20:32.482272+0000","last_became_active":"2024-04-25T11:20:31.454529+0000","last_became_peered":"2024-04-25T11:20:31.454529+0000","last_unstale":"2024-04-25T11:20:32.482272+0000","last_undegraded":"2024-04-25T11:20:32.482272+0000","last_fullsized":"2024-04-25T11:20:32.482272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:57.474821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.477078+0000","last_change":"2024-04-25T11:20:32.473893+0000","last_active":"2024-04-25T11:20:32.477078+0000","last_peered":"2024-04-25T11:20:32.477078+0000","last_clean":"2024-04-25T11:20:32.477078+0000","last_became_active":"2024-04-25T11:20:32.473178+0000","last_became_peered":"2024-04-25T11:20:32.473178+0000","last_unstale":"2024-04-25T11:20:32.477078+0000","last_undegraded":"2024-04-25T11:20:32.477078+0000","last_fullsized":"2024-04-25T11:20:32.477078+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:18.525729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413872+0000","last_change":"2024-04-25T11:20:31.443288+0000","last_active":"2024-04-25T11:20:33.413872+0000","last_peered":"2024-04-25T11:20:33.413872+0000","last_clean":"2024-04-25T11:20:33.413872+0000","last_became_active":"2024-04-25T11:20:31.442848+0000","last_became_peered":"2024-04-25T11:20:31.442848+0000","last_unstale":"2024-04-25T11:20:33.413872+0000","last_undegraded":"2024-04-25T11:20:33.413872+0000","last_fullsized":"2024-04-25T11:20:33.413872+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:50.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.436818+0000","last_change":"2024-04-25T11:20:32.432822+0000","last_active":"2024-04-25T11:20:32.436818+0000","last_peered":"2024-04-25T11:20:32.436818+0000","last_clean":"2024-04-25T11:20:32.436818+0000","last_became_active":"2024-04-25T11:20:32.431863+0000","last_became_peered":"2024-04-25T11:20:32.431863+0000","last_unstale":"2024-04-25T11:20:32.436818+0000","last_undegraded":"2024-04-25T11:20:32.436818+0000","last_fullsized":"2024-04-25T11:20:32.436818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:21.990942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902240+0000","last_change":"2024-04-25T11:20:31.426377+0000","last_active":"2024-04-25T11:20:33.902240+0000","last_peered":"2024-04-25T11:20:33.902240+0000","last_clean":"2024-04-25T11:20:33.902240+0000","last_became_active":"2024-04-25T11:20:31.425729+0000","last_became_peered":"2024-04-25T11:20:31.425729+0000","last_unstale":"2024-04-25T11:20:33.902240+0000","last_undegraded":"2024-04-25T11:20:33.902240+0000","last_fullsized":"2024-04-25T11:20:33.902240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:02.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.438299+0000","last_change":"2024-04-25T11:20:32.436365+0000","last_active":"2024-04-25T11:20:32.438299+0000","last_peered":"2024-04-25T11:20:32.438299+0000","last_clean":"2024-04-25T11:20:32.438299+0000","last_became_active":"2024-04-25T11:20:32.436046+0000","last_became_peered":"2024-04-25T11:20:32.436046+0000","last_unstale":"2024-04-25T11:20:32.438299+0000","last_undegraded":"2024-04-25T11:20:32.438299+0000","last_fullsized":"2024-04-25T11:20:32.438299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:03.896712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.416263+0000","last_change":"2024-04-25T11:20:31.426977+0000","last_active":"2024-04-25T11:20:33.416263+0000","last_peered":"2024-04-25T11:20:33.416263+0000","last_clean":"2024-04-25T11:20:33.416263+0000","last_became_active":"2024-04-25T11:20:31.426593+0000","last_became_peered":"2024-04-25T11:20:31.426593+0000","last_unstale":"2024-04-25T11:20:33.416263+0000","last_undegraded":"2024-04-25T11:20:33.416263+0000","last_fullsized":"2024-04-25T11:20:33.416263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:45.728761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479392+0000","last_change":"2024-04-25T11:20:32.473697+0000","last_active":"2024-04-25T11:20:32.479392+0000","last_peered":"2024-04-25T11:20:32.479392+0000","last_clean":"2024-04-25T11:20:32.479392+0000","last_became_active":"2024-04-25T11:20:32.472986+0000","last_became_peered":"2024-04-25T11:20:32.472986+0000","last_unstale":"2024-04-25T11:20:32.479392+0000","last_undegraded":"2024-04-25T11:20:32.479392+0000","last_fullsized":"2024-04-25T11:20:32.479392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.215661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.472547+0000","last_change":"2024-04-25T11:20:31.448054+0000","last_active":"2024-04-25T11:20:32.472547+0000","last_peered":"2024-04-25T11:20:32.472547+0000","last_clean":"2024-04-25T11:20:32.472547+0000","last_became_active":"2024-04-25T11:20:31.447194+0000","last_became_peered":"2024-04-25T11:20:31.447194+0000","last_unstale":"2024-04-25T11:20:32.472547+0000","last_undegraded":"2024-04-25T11:20:32.472547+0000","last_fullsized":"2024-04-25T11:20:32.472547+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:02.122639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.477951+0000","last_change":"2024-04-25T11:20:32.452317+0000","last_active":"2024-04-25T11:20:32.477951+0000","last_peered":"2024-04-25T11:20:32.477951+0000","last_clean":"2024-04-25T11:20:32.477951+0000","last_became_active":"2024-04-25T11:20:32.451578+0000","last_became_peered":"2024-04-25T11:20:32.451578+0000","last_unstale":"2024-04-25T11:20:32.477951+0000","last_undegraded":"2024-04-25T11:20:32.477951+0000","last_fullsized":"2024-04-25T11:20:32.477951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.471579+0000","last_change":"2024-04-25T11:20:31.445827+0000","last_active":"2024-04-25T11:20:32.471579+0000","last_peered":"2024-04-25T11:20:32.471579+0000","last_clean":"2024-04-25T11:20:32.471579+0000","last_became_active":"2024-04-25T11:20:31.444888+0000","last_became_peered":"2024-04-25T11:20:31.444888+0000","last_unstale":"2024-04-25T11:20:32.471579+0000","last_undegraded":"2024-04-25T11:20:32.471579+0000","last_fullsized":"2024-04-25T11:20:32.471579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:19.938517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.471484+0000","last_change":"2024-04-25T11:20:32.445482+0000","last_active":"2024-04-25T11:20:32.471484+0000","last_peered":"2024-04-25T11:20:32.471484+0000","last_clean":"2024-04-25T11:20:32.471484+0000","last_became_active":"2024-04-25T11:20:32.444461+0000","last_became_peered":"2024-04-25T11:20:32.444461+0000","last_unstale":"2024-04-25T11:20:32.471484+0000","last_undegraded":"2024-04-25T11:20:32.471484+0000","last_fullsized":"2024-04-25T11:20:32.471484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:03.896712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901424+0000","last_change":"2024-04-25T11:20:31.426671+0000","last_active":"2024-04-25T11:20:33.901424+0000","last_peered":"2024-04-25T11:20:33.901424+0000","last_clean":"2024-04-25T11:20:33.901424+0000","last_became_active":"2024-04-25T11:20:31.425832+0000","last_became_peered":"2024-04-25T11:20:31.425832+0000","last_unstale":"2024-04-25T11:20:33.901424+0000","last_undegraded":"2024-04-25T11:20:33.901424+0000","last_fullsized":"2024-04-25T11:20:33.901424+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:47.815189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.448685+0000","last_change":"2024-04-25T11:20:32.446484+0000","last_active":"2024-04-25T11:20:32.448685+0000","last_peered":"2024-04-25T11:20:32.448685+0000","last_clean":"2024-04-25T11:20:32.448685+0000","last_became_active":"2024-04-25T11:20:32.445645+0000","last_became_peered":"2024-04-25T11:20:32.445645+0000","last_unstale":"2024-04-25T11:20:32.448685+0000","last_undegraded":"2024-04-25T11:20:32.448685+0000","last_fullsized":"2024-04-25T11:20:32.448685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:11.840736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.415243+0000","last_change":"2024-04-25T11:20:31.436083+0000","last_active":"2024-04-25T11:20:32.415243+0000","last_peered":"2024-04-25T11:20:32.415243+0000","last_clean":"2024-04-25T11:20:32.415243+0000","last_became_active":"2024-04-25T11:20:31.435270+0000","last_became_peered":"2024-04-25T11:20:31.435270+0000","last_unstale":"2024-04-25T11:20:32.415243+0000","last_undegraded":"2024-04-25T11:20:32.415243+0000","last_fullsized":"2024-04-25T11:20:32.415243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:29.371839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.454737+0000","last_change":"2024-04-25T11:20:32.443409+0000","last_active":"2024-04-25T11:20:32.454737+0000","last_peered":"2024-04-25T11:20:32.454737+0000","last_clean":"2024-04-25T11:20:32.454737+0000","last_became_active":"2024-04-25T11:20:32.442118+0000","last_became_peered":"2024-04-25T11:20:32.442118+0000","last_unstale":"2024-04-25T11:20:32.454737+0000","last_undegraded":"2024-04-25T11:20:32.454737+0000","last_fullsized":"2024-04-25T11:20:32.454737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:03.896712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902695+0000","last_change":"2024-04-25T11:20:31.445363+0000","last_active":"2024-04-25T11:20:33.902695+0000","last_peered":"2024-04-25T11:20:33.902695+0000","last_clean":"2024-04-25T11:20:33.902695+0000","last_became_active":"2024-04-25T11:20:31.445037+0000","last_became_peered":"2024-04-25T11:20:31.445037+0000","last_unstale":"2024-04-25T11:20:33.902695+0000","last_undegraded":"2024-04-25T11:20:33.902695+0000","last_fullsized":"2024-04-25T11:20:33.902695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:28.640067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902715+0000","last_change":"2024-04-25T11:20:32.434340+0000","last_active":"2024-04-25T11:20:33.902715+0000","last_peered":"2024-04-25T11:20:33.902715+0000","last_clean":"2024-04-25T11:20:33.902715+0000","last_became_active":"2024-04-25T11:20:32.433962+0000","last_became_peered":"2024-04-25T11:20:32.433962+0000","last_unstale":"2024-04-25T11:20:33.902715+0000","last_undegraded":"2024-04-25T11:20:33.902715+0000","last_fullsized":"2024-04-25T11:20:33.902715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:35.636848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.414145+0000","last_change":"2024-04-25T11:20:31.424396+0000","last_active":"2024-04-25T11:20:33.414145+0000","last_peered":"2024-04-25T11:20:33.414145+0000","last_clean":"2024-04-25T11:20:33.414145+0000","last_became_active":"2024-04-25T11:20:31.423961+0000","last_became_peered":"2024-04-25T11:20:31.423961+0000","last_unstale":"2024-04-25T11:20:33.414145+0000","last_undegraded":"2024-04-25T11:20:33.414145+0000","last_fullsized":"2024-04-25T11:20:33.414145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:02.122639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902890+0000","last_change":"2024-04-25T11:20:32.453941+0000","last_active":"2024-04-25T11:20:33.902890+0000","last_peered":"2024-04-25T11:20:33.902890+0000","last_clean":"2024-04-25T11:20:33.902890+0000","last_became_active":"2024-04-25T11:20:32.453673+0000","last_became_peered":"2024-04-25T11:20:32.453673+0000","last_unstale":"2024-04-25T11:20:33.902890+0000","last_undegraded":"2024-04-25T11:20:33.902890+0000","last_fullsized":"2024-04-25T11:20:33.902890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:15.873495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.482526+0000","last_change":"2024-04-25T11:20:31.449797+0000","last_active":"2024-04-25T11:20:32.482526+0000","last_peered":"2024-04-25T11:20:32.482526+0000","last_clean":"2024-04-25T11:20:32.482526+0000","last_became_active":"2024-04-25T11:20:31.448637+0000","last_became_peered":"2024-04-25T11:20:31.448637+0000","last_unstale":"2024-04-25T11:20:32.482526+0000","last_undegraded":"2024-04-25T11:20:32.482526+0000","last_fullsized":"2024-04-25T11:20:32.482526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:53.828515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.481749+0000","last_change":"2024-04-25T11:20:32.460594+0000","last_active":"2024-04-25T11:20:32.481749+0000","last_peered":"2024-04-25T11:20:32.481749+0000","last_clean":"2024-04-25T11:20:32.481749+0000","last_became_active":"2024-04-25T11:20:32.457327+0000","last_became_peered":"2024-04-25T11:20:32.457327+0000","last_unstale":"2024-04-25T11:20:32.481749+0000","last_undegraded":"2024-04-25T11:20:32.481749+0000","last_fullsized":"2024-04-25T11:20:32.481749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.215661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902841+0000","last_change":"2024-04-25T11:20:31.427092+0000","last_active":"2024-04-25T11:20:33.902841+0000","last_peered":"2024-04-25T11:20:33.902841+0000","last_clean":"2024-04-25T11:20:33.902841+0000","last_became_active":"2024-04-25T11:20:31.426396+0000","last_became_peered":"2024-04-25T11:20:31.426396+0000","last_unstale":"2024-04-25T11:20:33.902841+0000","last_undegraded":"2024-04-25T11:20:33.902841+0000","last_fullsized":"2024-04-25T11:20:33.902841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:33.759006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.449056+0000","last_change":"2024-04-25T11:20:32.446126+0000","last_active":"2024-04-25T11:20:32.449056+0000","last_peered":"2024-04-25T11:20:32.449056+0000","last_clean":"2024-04-25T11:20:32.449056+0000","last_became_active":"2024-04-25T11:20:32.445725+0000","last_became_peered":"2024-04-25T11:20:32.445725+0000","last_unstale":"2024-04-25T11:20:32.449056+0000","last_undegraded":"2024-04-25T11:20:32.449056+0000","last_fullsized":"2024-04-25T11:20:32.449056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:29.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.472440+0000","last_change":"2024-04-25T11:20:31.446205+0000","last_active":"2024-04-25T11:20:32.472440+0000","last_peered":"2024-04-25T11:20:32.472440+0000","last_clean":"2024-04-25T11:20:32.472440+0000","last_became_active":"2024-04-25T11:20:31.443131+0000","last_became_peered":"2024-04-25T11:20:31.443131+0000","last_unstale":"2024-04-25T11:20:32.472440+0000","last_undegraded":"2024-04-25T11:20:32.472440+0000","last_fullsized":"2024-04-25T11:20:32.472440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:29.591167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902491+0000","last_change":"2024-04-25T11:20:32.425087+0000","last_active":"2024-04-25T11:20:33.902491+0000","last_peered":"2024-04-25T11:20:33.902491+0000","last_clean":"2024-04-25T11:20:33.902491+0000","last_became_active":"2024-04-25T11:20:32.424703+0000","last_became_peered":"2024-04-25T11:20:32.424703+0000","last_unstale":"2024-04-25T11:20:33.902491+0000","last_undegraded":"2024-04-25T11:20:33.902491+0000","last_fullsized":"2024-04-25T11:20:33.902491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:23.526725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.423168+0000","last_change":"2024-04-25T11:20:31.435320+0000","last_active":"2024-04-25T11:20:32.423168+0000","last_peered":"2024-04-25T11:20:32.423168+0000","last_clean":"2024-04-25T11:20:32.423168+0000","last_became_active":"2024-04-25T11:20:31.434942+0000","last_became_peered":"2024-04-25T11:20:31.434942+0000","last_unstale":"2024-04-25T11:20:32.423168+0000","last_undegraded":"2024-04-25T11:20:32.423168+0000","last_fullsized":"2024-04-25T11:20:32.423168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:57.474821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.480543+0000","last_change":"2024-04-25T11:20:32.477985+0000","last_active":"2024-04-25T11:20:32.480543+0000","last_peered":"2024-04-25T11:20:32.480543+0000","last_clean":"2024-04-25T11:20:32.480543+0000","last_became_active":"2024-04-25T11:20:32.477365+0000","last_became_peered":"2024-04-25T11:20:32.477365+0000","last_unstale":"2024-04-25T11:20:32.480543+0000","last_undegraded":"2024-04-25T11:20:32.480543+0000","last_fullsized":"2024-04-25T11:20:32.480543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:54.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902464+0000","last_change":"2024-04-25T11:20:31.445398+0000","last_active":"2024-04-25T11:20:33.902464+0000","last_peered":"2024-04-25T11:20:33.902464+0000","last_clean":"2024-04-25T11:20:33.902464+0000","last_became_active":"2024-04-25T11:20:31.445020+0000","last_became_peered":"2024-04-25T11:20:31.445020+0000","last_unstale":"2024-04-25T11:20:33.902464+0000","last_undegraded":"2024-04-25T11:20:33.902464+0000","last_fullsized":"2024-04-25T11:20:33.902464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:53.828515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.478539+0000","last_change":"2024-04-25T11:20:32.475810+0000","last_active":"2024-04-25T11:20:32.478539+0000","last_peered":"2024-04-25T11:20:32.478539+0000","last_clean":"2024-04-25T11:20:32.478539+0000","last_became_active":"2024-04-25T11:20:32.474934+0000","last_became_peered":"2024-04-25T11:20:32.474934+0000","last_unstale":"2024-04-25T11:20:32.478539+0000","last_undegraded":"2024-04-25T11:20:32.478539+0000","last_fullsized":"2024-04-25T11:20:32.478539+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901566+0000","last_change":"2024-04-25T11:20:31.429431+0000","last_active":"2024-04-25T11:20:33.901566+0000","last_peered":"2024-04-25T11:20:33.901566+0000","last_clean":"2024-04-25T11:20:33.901566+0000","last_became_active":"2024-04-25T11:20:31.428993+0000","last_became_peered":"2024-04-25T11:20:31.428993+0000","last_unstale":"2024-04-25T11:20:33.901566+0000","last_undegraded":"2024-04-25T11:20:33.901566+0000","last_fullsized":"2024-04-25T11:20:33.901566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:21.148124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.471932+0000","last_change":"2024-04-25T11:20:32.455223+0000","last_active":"2024-04-25T11:20:32.471932+0000","last_peered":"2024-04-25T11:20:32.471932+0000","last_clean":"2024-04-25T11:20:32.471932+0000","last_became_active":"2024-04-25T11:20:32.454296+0000","last_became_peered":"2024-04-25T11:20:32.454296+0000","last_unstale":"2024-04-25T11:20:32.471932+0000","last_undegraded":"2024-04-25T11:20:32.471932+0000","last_fullsized":"2024-04-25T11:20:32.471932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.440305+0000","last_change":"2024-04-25T11:20:31.429207+0000","last_active":"2024-04-25T11:20:32.440305+0000","last_peered":"2024-04-25T11:20:32.440305+0000","last_clean":"2024-04-25T11:20:32.440305+0000","last_became_active":"2024-04-25T11:20:31.428781+0000","last_became_peered":"2024-04-25T11:20:31.428781+0000","last_unstale":"2024-04-25T11:20:32.440305+0000","last_undegraded":"2024-04-25T11:20:32.440305+0000","last_fullsized":"2024-04-25T11:20:32.440305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:29.591167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.482320+0000","last_change":"2024-04-25T11:20:32.474150+0000","last_active":"2024-04-25T11:20:32.482320+0000","last_peered":"2024-04-25T11:20:32.482320+0000","last_clean":"2024-04-25T11:20:32.482320+0000","last_became_active":"2024-04-25T11:20:32.473578+0000","last_became_peered":"2024-04-25T11:20:32.473578+0000","last_unstale":"2024-04-25T11:20:32.482320+0000","last_undegraded":"2024-04-25T11:20:32.482320+0000","last_fullsized":"2024-04-25T11:20:32.482320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:54.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.469254+0000","last_change":"2024-04-25T11:20:31.445112+0000","last_active":"2024-04-25T11:20:32.469254+0000","last_peered":"2024-04-25T11:20:32.469254+0000","last_clean":"2024-04-25T11:20:32.469254+0000","last_became_active":"2024-04-25T11:20:31.444214+0000","last_became_peered":"2024-04-25T11:20:31.444214+0000","last_unstale":"2024-04-25T11:20:32.469254+0000","last_undegraded":"2024-04-25T11:20:32.469254+0000","last_fullsized":"2024-04-25T11:20:32.469254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:18.138532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.438338+0000","last_change":"2024-04-25T11:20:32.432742+0000","last_active":"2024-04-25T11:20:32.438338+0000","last_peered":"2024-04-25T11:20:32.438338+0000","last_clean":"2024-04-25T11:20:32.438338+0000","last_became_active":"2024-04-25T11:20:32.432297+0000","last_became_peered":"2024-04-25T11:20:32.432297+0000","last_unstale":"2024-04-25T11:20:32.438338+0000","last_undegraded":"2024-04-25T11:20:32.438338+0000","last_fullsized":"2024-04-25T11:20:32.438338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:22.158494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.472101+0000","last_change":"2024-04-25T11:20:31.444332+0000","last_active":"2024-04-25T11:20:32.472101+0000","last_peered":"2024-04-25T11:20:32.472101+0000","last_clean":"2024-04-25T11:20:32.472101+0000","last_became_active":"2024-04-25T11:20:31.443377+0000","last_became_peered":"2024-04-25T11:20:31.443377+0000","last_unstale":"2024-04-25T11:20:32.472101+0000","last_undegraded":"2024-04-25T11:20:32.472101+0000","last_fullsized":"2024-04-25T11:20:32.472101+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:57.474821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415974+0000","last_change":"2024-04-25T11:20:32.450092+0000","last_active":"2024-04-25T11:20:33.415974+0000","last_peered":"2024-04-25T11:20:33.415974+0000","last_clean":"2024-04-25T11:20:33.415974+0000","last_became_active":"2024-04-25T11:20:32.449702+0000","last_became_peered":"2024-04-25T11:20:32.449702+0000","last_unstale":"2024-04-25T11:20:33.415974+0000","last_undegraded":"2024-04-25T11:20:33.415974+0000","last_fullsized":"2024-04-25T11:20:33.415974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:18.525729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.416246+0000","last_change":"2024-04-25T11:20:31.431649+0000","last_active":"2024-04-25T11:20:32.416246+0000","last_peered":"2024-04-25T11:20:32.416246+0000","last_clean":"2024-04-25T11:20:32.416246+0000","last_became_active":"2024-04-25T11:20:31.430754+0000","last_became_peered":"2024-04-25T11:20:31.430754+0000","last_unstale":"2024-04-25T11:20:32.416246+0000","last_undegraded":"2024-04-25T11:20:32.416246+0000","last_fullsized":"2024-04-25T11:20:32.416246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:44.072745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.416184+0000","last_change":"2024-04-25T11:20:32.429410+0000","last_active":"2024-04-25T11:20:33.416184+0000","last_peered":"2024-04-25T11:20:33.416184+0000","last_clean":"2024-04-25T11:20:33.416184+0000","last_became_active":"2024-04-25T11:20:32.429082+0000","last_became_peered":"2024-04-25T11:20:32.429082+0000","last_unstale":"2024-04-25T11:20:33.416184+0000","last_undegraded":"2024-04-25T11:20:33.416184+0000","last_fullsized":"2024-04-25T11:20:33.416184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413920+0000","last_change":"2024-04-25T11:20:31.421932+0000","last_active":"2024-04-25T11:20:33.413920+0000","last_peered":"2024-04-25T11:20:33.413920+0000","last_clean":"2024-04-25T11:20:33.413920+0000","last_became_active":"2024-04-25T11:20:31.421590+0000","last_became_peered":"2024-04-25T11:20:31.421590+0000","last_unstale":"2024-04-25T11:20:33.413920+0000","last_undegraded":"2024-04-25T11:20:33.413920+0000","last_fullsized":"2024-04-25T11:20:33.413920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:57.474821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415644+0000","last_change":"2024-04-25T11:20:32.431199+0000","last_active":"2024-04-25T11:20:33.415644+0000","last_peered":"2024-04-25T11:20:33.415644+0000","last_clean":"2024-04-25T11:20:33.415644+0000","last_became_active":"2024-04-25T11:20:32.430607+0000","last_became_peered":"2024-04-25T11:20:32.430607+0000","last_unstale":"2024-04-25T11:20:33.415644+0000","last_undegraded":"2024-04-25T11:20:33.415644+0000","last_fullsized":"2024-04-25T11:20:33.415644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:58.485191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.438137+0000","last_change":"2024-04-25T11:20:31.442153+0000","last_active":"2024-04-25T11:20:32.438137+0000","last_peered":"2024-04-25T11:20:32.438137+0000","last_clean":"2024-04-25T11:20:32.438137+0000","last_became_active":"2024-04-25T11:20:31.441192+0000","last_became_peered":"2024-04-25T11:20:31.441192+0000","last_unstale":"2024-04-25T11:20:32.438137+0000","last_undegraded":"2024-04-25T11:20:32.438137+0000","last_fullsized":"2024-04-25T11:20:32.438137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:18.138532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415227+0000","last_change":"2024-04-25T11:20:32.427267+0000","last_active":"2024-04-25T11:20:33.415227+0000","last_peered":"2024-04-25T11:20:33.415227+0000","last_clean":"2024-04-25T11:20:33.415227+0000","last_became_active":"2024-04-25T11:20:32.426961+0000","last_became_peered":"2024-04-25T11:20:32.426961+0000","last_unstale":"2024-04-25T11:20:33.415227+0000","last_undegraded":"2024-04-25T11:20:33.415227+0000","last_fullsized":"2024-04-25T11:20:33.415227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:45.083115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.415921+0000","last_change":"2024-04-25T11:20:31.445706+0000","last_active":"2024-04-25T11:20:32.415921+0000","last_peered":"2024-04-25T11:20:32.415921+0000","last_clean":"2024-04-25T11:20:32.415921+0000","last_became_active":"2024-04-25T11:20:31.443158+0000","last_became_peered":"2024-04-25T11:20:31.443158+0000","last_unstale":"2024-04-25T11:20:32.415921+0000","last_undegraded":"2024-04-25T11:20:32.415921+0000","last_fullsized":"2024-04-25T11:20:32.415921+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:21.456266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479409+0000","last_change":"2024-04-25T11:20:32.473907+0000","last_active":"2024-04-25T11:20:32.479409+0000","last_peered":"2024-04-25T11:20:32.479409+0000","last_clean":"2024-04-25T11:20:32.479409+0000","last_became_active":"2024-04-25T11:20:32.472640+0000","last_became_peered":"2024-04-25T11:20:32.472640+0000","last_unstale":"2024-04-25T11:20:32.479409+0000","last_undegraded":"2024-04-25T11:20:32.479409+0000","last_fullsized":"2024-04-25T11:20:32.479409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:34.139129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.436706+0000","last_change":"2024-04-25T11:20:31.454488+0000","last_active":"2024-04-25T11:20:32.436706+0000","last_peered":"2024-04-25T11:20:32.436706+0000","last_clean":"2024-04-25T11:20:32.436706+0000","last_became_active":"2024-04-25T11:20:31.453689+0000","last_became_peered":"2024-04-25T11:20:31.453689+0000","last_unstale":"2024-04-25T11:20:32.436706+0000","last_undegraded":"2024-04-25T11:20:32.436706+0000","last_fullsized":"2024-04-25T11:20:32.436706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:20.980572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415969+0000","last_change":"2024-04-25T11:20:32.463198+0000","last_active":"2024-04-25T11:20:33.415969+0000","last_peered":"2024-04-25T11:20:33.415969+0000","last_clean":"2024-04-25T11:20:33.415969+0000","last_became_active":"2024-04-25T11:20:32.462875+0000","last_became_peered":"2024-04-25T11:20:32.462875+0000","last_unstale":"2024-04-25T11:20:33.415969+0000","last_undegraded":"2024-04-25T11:20:33.415969+0000","last_fullsized":"2024-04-25T11:20:33.415969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:30.601537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.478184+0000","last_change":"2024-04-25T11:20:31.452982+0000","last_active":"2024-04-25T11:20:32.478184+0000","last_peered":"2024-04-25T11:20:32.478184+0000","last_clean":"2024-04-25T11:20:32.478184+0000","last_became_active":"2024-04-25T11:20:31.452614+0000","last_became_peered":"2024-04-25T11:20:31.452614+0000","last_unstale":"2024-04-25T11:20:32.478184+0000","last_undegraded":"2024-04-25T11:20:32.478184+0000","last_fullsized":"2024-04-25T11:20:32.478184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:17.515359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.478764+0000","last_change":"2024-04-25T11:20:32.456269+0000","last_active":"2024-04-25T11:20:32.478764+0000","last_peered":"2024-04-25T11:20:32.478764+0000","last_clean":"2024-04-25T11:20:32.478764+0000","last_became_active":"2024-04-25T11:20:32.455367+0000","last_became_peered":"2024-04-25T11:20:32.455367+0000","last_unstale":"2024-04-25T11:20:32.478764+0000","last_undegraded":"2024-04-25T11:20:32.478764+0000","last_fullsized":"2024-04-25T11:20:32.478764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.006165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.414431+0000","last_change":"2024-04-25T11:20:31.418753+0000","last_active":"2024-04-25T11:20:33.414431+0000","last_peered":"2024-04-25T11:20:33.414431+0000","last_clean":"2024-04-25T11:20:33.414431+0000","last_became_active":"2024-04-25T11:20:31.418381+0000","last_became_peered":"2024-04-25T11:20:31.418381+0000","last_unstale":"2024-04-25T11:20:33.414431+0000","last_undegraded":"2024-04-25T11:20:33.414431+0000","last_fullsized":"2024-04-25T11:20:33.414431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:17.515359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413308+0000","last_change":"2024-04-25T11:20:32.427144+0000","last_active":"2024-04-25T11:20:33.413308+0000","last_peered":"2024-04-25T11:20:33.413308+0000","last_clean":"2024-04-25T11:20:33.413308+0000","last_became_active":"2024-04-25T11:20:32.426765+0000","last_became_peered":"2024-04-25T11:20:32.426765+0000","last_unstale":"2024-04-25T11:20:33.413308+0000","last_undegraded":"2024-04-25T11:20:33.413308+0000","last_fullsized":"2024-04-25T11:20:33.413308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.423780+0000","last_change":"2024-04-25T11:20:31.422133+0000","last_active":"2024-04-25T11:20:32.423780+0000","last_peered":"2024-04-25T11:20:32.423780+0000","last_clean":"2024-04-25T11:20:32.423780+0000","last_became_active":"2024-04-25T11:20:31.421558+0000","last_became_peered":"2024-04-25T11:20:31.421558+0000","last_unstale":"2024-04-25T11:20:32.423780+0000","last_undegraded":"2024-04-25T11:20:32.423780+0000","last_fullsized":"2024-04-25T11:20:32.423780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:02.122639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.467364+0000","last_change":"2024-04-25T11:20:32.465303+0000","last_active":"2024-04-25T11:20:32.467364+0000","last_peered":"2024-04-25T11:20:32.467364+0000","last_clean":"2024-04-25T11:20:32.467364+0000","last_became_active":"2024-04-25T11:20:32.464414+0000","last_became_peered":"2024-04-25T11:20:32.464414+0000","last_unstale":"2024-04-25T11:20:32.467364+0000","last_undegraded":"2024-04-25T11:20:32.467364+0000","last_fullsized":"2024-04-25T11:20:32.467364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:51.320902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.414175+0000","last_change":"2024-04-25T11:20:31.439237+0000","last_active":"2024-04-25T11:20:33.414175+0000","last_peered":"2024-04-25T11:20:33.414175+0000","last_clean":"2024-04-25T11:20:33.414175+0000","last_became_active":"2024-04-25T11:20:31.438862+0000","last_became_peered":"2024-04-25T11:20:31.438862+0000","last_unstale":"2024-04-25T11:20:33.414175+0000","last_undegraded":"2024-04-25T11:20:33.414175+0000","last_fullsized":"2024-04-25T11:20:33.414175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:29.591167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413758+0000","last_change":"2024-04-25T11:20:32.438678+0000","last_active":"2024-04-25T11:20:33.413758+0000","last_peered":"2024-04-25T11:20:33.413758+0000","last_clean":"2024-04-25T11:20:33.413758+0000","last_became_active":"2024-04-25T11:20:32.438342+0000","last_became_peered":"2024-04-25T11:20:32.438342+0000","last_unstale":"2024-04-25T11:20:33.413758+0000","last_undegraded":"2024-04-25T11:20:33.413758+0000","last_fullsized":"2024-04-25T11:20:33.413758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:11.840736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.423372+0000","last_change":"2024-04-25T11:20:31.438307+0000","last_active":"2024-04-25T11:20:32.423372+0000","last_peered":"2024-04-25T11:20:32.423372+0000","last_clean":"2024-04-25T11:20:32.423372+0000","last_became_active":"2024-04-25T11:20:31.437983+0000","last_became_peered":"2024-04-25T11:20:31.437983+0000","last_unstale":"2024-04-25T11:20:32.423372+0000","last_undegraded":"2024-04-25T11:20:32.423372+0000","last_fullsized":"2024-04-25T11:20:32.423372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:50.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901484+0000","last_change":"2024-04-25T11:20:32.427095+0000","last_active":"2024-04-25T11:20:33.901484+0000","last_peered":"2024-04-25T11:20:33.901484+0000","last_clean":"2024-04-25T11:20:33.901484+0000","last_became_active":"2024-04-25T11:20:32.426584+0000","last_became_peered":"2024-04-25T11:20:32.426584+0000","last_unstale":"2024-04-25T11:20:33.901484+0000","last_undegraded":"2024-04-25T11:20:33.901484+0000","last_fullsized":"2024-04-25T11:20:33.901484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:10.846516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.415957+0000","last_change":"2024-04-25T11:20:31.454753+0000","last_active":"2024-04-25T11:20:32.415957+0000","last_peered":"2024-04-25T11:20:32.415957+0000","last_clean":"2024-04-25T11:20:32.415957+0000","last_became_active":"2024-04-25T11:20:31.453969+0000","last_became_peered":"2024-04-25T11:20:31.453969+0000","last_unstale":"2024-04-25T11:20:32.415957+0000","last_undegraded":"2024-04-25T11:20:32.415957+0000","last_fullsized":"2024-04-25T11:20:32.415957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:03.172347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.428251+0000","last_change":"2024-04-25T11:20:32.420333+0000","last_active":"2024-04-25T11:20:32.428251+0000","last_peered":"2024-04-25T11:20:32.428251+0000","last_clean":"2024-04-25T11:20:32.428251+0000","last_became_active":"2024-04-25T11:20:32.419345+0000","last_became_peered":"2024-04-25T11:20:32.419345+0000","last_unstale":"2024-04-25T11:20:32.428251+0000","last_undegraded":"2024-04-25T11:20:32.428251+0000","last_fullsized":"2024-04-25T11:20:32.428251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:51.320902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901789+0000","last_change":"2024-04-25T11:20:31.446124+0000","last_active":"2024-04-25T11:20:33.901789+0000","last_peered":"2024-04-25T11:20:33.901789+0000","last_clean":"2024-04-25T11:20:33.901789+0000","last_became_active":"2024-04-25T11:20:31.445804+0000","last_became_peered":"2024-04-25T11:20:31.445804+0000","last_unstale":"2024-04-25T11:20:33.901789+0000","last_undegraded":"2024-04-25T11:20:33.901789+0000","last_fullsized":"2024-04-25T11:20:33.901789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:20.980572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.470297+0000","last_change":"2024-04-25T11:20:32.463197+0000","last_active":"2024-04-25T11:20:32.470297+0000","last_peered":"2024-04-25T11:20:32.470297+0000","last_clean":"2024-04-25T11:20:32.470297+0000","last_became_active":"2024-04-25T11:20:32.453859+0000","last_became_peered":"2024-04-25T11:20:32.453859+0000","last_unstale":"2024-04-25T11:20:32.470297+0000","last_undegraded":"2024-04-25T11:20:32.470297+0000","last_fullsized":"2024-04-25T11:20:32.470297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:29.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.437075+0000","last_change":"2024-04-25T11:20:31.445115+0000","last_active":"2024-04-25T11:20:32.437075+0000","last_peered":"2024-04-25T11:20:32.437075+0000","last_clean":"2024-04-25T11:20:32.437075+0000","last_became_active":"2024-04-25T11:20:31.444279+0000","last_became_peered":"2024-04-25T11:20:31.444279+0000","last_unstale":"2024-04-25T11:20:32.437075+0000","last_undegraded":"2024-04-25T11:20:32.437075+0000","last_fullsized":"2024-04-25T11:20:32.437075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:44.072745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":132,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.453166+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:20:32.453166+0000","last_peered":"2024-04-25T11:20:32.453166+0000","last_clean":"2024-04-25T11:20:32.453166+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:20:32.453166+0000","last_undegraded":"2024-04-25T11:20:32.453166+0000","last_fullsized":"2024-04-25T11:20:32.453166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:20:36.586 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:20:36.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5756736640 1 Processor -- start 2024-04-25T11:20:36.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5756736640 1 -- start start 2024-04-25T11:20:36.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5756736640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f575015e0a0 0x7f5750073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5756736640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 0x7f57500cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5756736640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57500cf1a0 con 0x7f575006f5f0 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5756736640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f57500d1cf0 con 0x7f575015e0a0 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5756736640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f57500ce050 con 0x7f57500ce850 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5754cd3640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f575006f5f0 0x7f575006fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52566/0 (socket says 172.21.15.106:52566) 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5754cd3640 1 -- 172.21.15.106:0/2147481871 learned_addr learned my addr 172.21.15.106:0/2147481871 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347781430 0 0) 0x7f57400014a0 con 0x7f575006f5f0 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5738003530 con 0x7f575006f5f0 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574f7fe640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 0x7f57500cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574ffff640 1 --2- 172.21.15.106:0/2147481871 >> v2:172.21.15.134:3300/0 conn(0x7f575015e0a0 0x7f5750073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2861810500 0 0) 0x7f5738003530 con 0x7f575006f5f0 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 msgr2=0x7f57500cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 0x7f57500cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 >> v2:172.21.15.134:3300/0 conn(0x7f575015e0a0 msgr2=0x7f5750073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 --2- 172.21.15.106:0/2147481871 >> v2:172.21.15.134:3300/0 conn(0x7f575015e0a0 0x7f5750073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575015e6b0 con 0x7f575006f5f0 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f5740001bd0 con 0x7f575006f5f0 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574f7fe640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 0x7f57500cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 >> v1:172.21.15.106:6789/0 conn(0x7f575006f5f0 legacy=0x7f575006fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 0x7f57380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738003fd0 0x7f5738004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738004d50 0x7f5738005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57500ce050 con 0x7f5738004d50 2024-04-25T11:20:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f573800bae0 con 0x7f5738003fd0 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f573800bce0 con 0x7f57500ce850 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f5754cd3640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738004d50 0x7f5738005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574ffff640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 0x7f57380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.771+0000 7f574f7fe640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738003fd0 0x7f5738004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5754cd3640 1 -- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 msgr2=0x7f57380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5754cd3640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57500ce850 0x7f57380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5754cd3640 1 -- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738003fd0 msgr2=0x7f5738004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5754cd3640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738003fd0 0x7f5738004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5754cd3640 1 -- 172.21.15.106:0/2147481871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5738003530 con 0x7f5738004d50 2024-04-25T11:20:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5754cd3640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738004d50 0x7f5738005160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5750070ae0 tx=0x7f574001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f65f5217fe9c483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574000a670 con 0x7f5738004d50 2024-04-25T11:20:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57400ad650 con 0x7f5738004d50 2024-04-25T11:20:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f574effd640 1 -- 172.21.15.106:0/2147481871 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57400ada80 con 0x7f5738004d50 2024-04-25T11:20:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5756736640 1 -- 172.21.15.106:0/2147481871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738004d50 msgr2=0x7f5738005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5756736640 1 --2- 172.21.15.106:0/2147481871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738004d50 0x7f5738005160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5750070ae0 tx=0x7f574001d340 comp rx=0 tx=0).stop 2024-04-25T11:20:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5754cd3640 1 -- 172.21.15.106:0/2147481871 reap_dead start 2024-04-25T11:20:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5756736640 1 -- 172.21.15.106:0/2147481871 shutdown_connections 2024-04-25T11:20:36.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5756736640 1 -- 172.21.15.106:0/2147481871 >> 172.21.15.106:0/2147481871 conn(0x7f575006c7c0 msgr2=0x7f575006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5756736640 1 -- 172.21.15.106:0/2147481871 shutdown_connections 2024-04-25T11:20:36.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.775+0000 7f5756736640 1 -- 172.21.15.106:0/2147481871 wait complete. 2024-04-25T11:20:36.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 Processor -- start 2024-04-25T11:20:36.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 -- start start 2024-04-25T11:20:36.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738004d50 0x7f575013e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5750151370 0x7f57501f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57501f7720 0x7f575013bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f575015ef50 con 0x7f57501f7720 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f575015ec70 con 0x7f5738004d50 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f57500d06e0 con 0x7f5750151370 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57501f7720 0x7f575013bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57501f7720 0x7f575013bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60310/0 (socket says 172.21.15.106:60310) 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 -- 172.21.15.106:0/3770359648 learned_addr learned my addr 172.21.15.106:0/3770359648 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f574ffff640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738004d50 0x7f575013e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f574f7fe640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5750151370 0x7f57501f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 -- 172.21.15.106:0/3770359648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5750151370 msgr2=0x7f57501f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5750151370 0x7f57501f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 -- 172.21.15.106:0/3770359648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738004d50 msgr2=0x7f575013e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738004d50 0x7f575013e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 -- 172.21.15.106:0/3770359648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57501e53f0 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5754cd3640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57501f7720 0x7f575013bd70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f574001cdb0 tx=0x7f5738005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f574cff9640 1 -- 172.21.15.106:0/3770359648 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57400bc070 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f574cff9640 1 -- 172.21.15.106:0/3770359648 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57400011d0 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57501e56d0 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f574cff9640 1 -- 172.21.15.106:0/3770359648 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574000a670 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575015d920 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.779+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57501f7fa0 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.783+0000 7f574cff9640 1 -- 172.21.15.106:0/3770359648 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57400b5070 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.783+0000 7f574cff9640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f573c06f430 0x7f573c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.783+0000 7f574ffff640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f573c06f430 0x7f573c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.783+0000 7f574cff9640 1 -- 172.21.15.106:0/3770359648 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5740128d00 con 0x7f57501f7720 2024-04-25T11:20:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.783+0000 7f574ffff640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f573c06f430 0x7f573c071940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f574800ba30 tx=0x7f5748008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.787+0000 7f574cff9640 1 -- 172.21.15.106:0/3770359648 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57500cf1a0 con 0x7f57501f7720 2024-04-25T11:20:36.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.931+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f575006cbd0 con 0x7f573c06f430 2024-04-25T11:20:36.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.939+0000 7f574cff9640 1 -- 172.21.15.106:0/3770359648 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251901 (secure 0 0 0) 0x7f575006cbd0 con 0x7f573c06f430 2024-04-25T11:20:36.940 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:36.942 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f573c06f430 msgr2=0x7f573c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f573c06f430 0x7f573c071940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f574800ba30 tx=0x7f5748008040 comp rx=0 tx=0).stop 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57501f7720 msgr2=0x7f575013bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57501f7720 0x7f575013bd70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f574001cdb0 tx=0x7f5738005840 comp rx=0 tx=0).stop 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 shutdown_connections 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f573c06f430 0x7f573c071940 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57501f7720 0x7f575013bd70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5750151370 0x7f57501f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 --2- 172.21.15.106:0/3770359648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5738004d50 0x7f575013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 >> 172.21.15.106:0/3770359648 conn(0x7f575006c7c0 msgr2=0x7f57500d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 shutdown_connections 2024-04-25T11:20:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:36.943+0000 7f5756736640 1 -- 172.21.15.106:0/3770359648 wait complete. 2024-04-25T11:20:36.960 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T11:20:36.113546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220232,"kb_used_data":4064,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754488,"statfs":{"total":773094113280,"available":772868595712,"internally_reserved":0,"allocated":4161536,"data_stored":2320620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.720323"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.474260+0000","last_change":"2024-04-25T11:20:32.465790+0000","last_active":"2024-04-25T11:20:32.474260+0000","last_peered":"2024-04-25T11:20:32.474260+0000","last_clean":"2024-04-25T11:20:32.474260+0000","last_became_active":"2024-04-25T11:20:32.459298+0000","last_became_peered":"2024-04-25T11:20:32.459298+0000","last_unstale":"2024-04-25T11:20:32.474260+0000","last_undegraded":"2024-04-25T11:20:32.474260+0000","last_fullsized":"2024-04-25T11:20:32.474260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:03.133009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902433+0000","last_change":"2024-04-25T11:20:32.428769+0000","last_active":"2024-04-25T11:20:33.902433+0000","last_peered":"2024-04-25T11:20:33.902433+0000","last_clean":"2024-04-25T11:20:33.902433+0000","last_became_active":"2024-04-25T11:20:32.428420+0000","last_became_peered":"2024-04-25T11:20:32.428420+0000","last_unstale":"2024-04-25T11:20:33.902433+0000","last_undegraded":"2024-04-25T11:20:33.902433+0000","last_fullsized":"2024-04-25T11:20:33.902433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:47.880494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.478949+0000","last_change":"2024-04-25T11:20:32.456294+0000","last_active":"2024-04-25T11:20:32.478949+0000","last_peered":"2024-04-25T11:20:32.478949+0000","last_clean":"2024-04-25T11:20:32.478949+0000","last_became_active":"2024-04-25T11:20:32.455501+0000","last_became_peered":"2024-04-25T11:20:32.455501+0000","last_unstale":"2024-04-25T11:20:32.478949+0000","last_undegraded":"2024-04-25T11:20:32.478949+0000","last_fullsized":"2024-04-25T11:20:32.478949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:56.022841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.441235+0000","last_change":"2024-04-25T11:20:32.440116+0000","last_active":"2024-04-25T11:20:32.441235+0000","last_peered":"2024-04-25T11:20:32.441235+0000","last_clean":"2024-04-25T11:20:32.441235+0000","last_became_active":"2024-04-25T11:20:32.439588+0000","last_became_peered":"2024-04-25T11:20:32.439588+0000","last_unstale":"2024-04-25T11:20:32.441235+0000","last_undegraded":"2024-04-25T11:20:32.441235+0000","last_fullsized":"2024-04-25T11:20:32.441235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415470+0000","last_change":"2024-04-25T11:20:32.438053+0000","last_active":"2024-04-25T11:20:33.415470+0000","last_peered":"2024-04-25T11:20:33.415470+0000","last_clean":"2024-04-25T11:20:33.415470+0000","last_became_active":"2024-04-25T11:20:32.437359+0000","last_became_peered":"2024-04-25T11:20:32.437359+0000","last_unstale":"2024-04-25T11:20:33.415470+0000","last_undegraded":"2024-04-25T11:20:33.415470+0000","last_fullsized":"2024-04-25T11:20:33.415470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:19.148902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901892+0000","last_change":"2024-04-25T11:20:32.434766+0000","last_active":"2024-04-25T11:20:33.901892+0000","last_peered":"2024-04-25T11:20:33.901892+0000","last_clean":"2024-04-25T11:20:33.901892+0000","last_became_active":"2024-04-25T11:20:32.434350+0000","last_became_peered":"2024-04-25T11:20:32.434350+0000","last_unstale":"2024-04-25T11:20:33.901892+0000","last_undegraded":"2024-04-25T11:20:33.901892+0000","last_fullsized":"2024-04-25T11:20:33.901892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:56.806426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.470968+0000","last_change":"2024-04-25T11:20:32.469167+0000","last_active":"2024-04-25T11:20:32.470968+0000","last_peered":"2024-04-25T11:20:32.470968+0000","last_clean":"2024-04-25T11:20:32.470968+0000","last_became_active":"2024-04-25T11:20:32.468381+0000","last_became_peered":"2024-04-25T11:20:32.468381+0000","last_unstale":"2024-04-25T11:20:32.470968+0000","last_undegraded":"2024-04-25T11:20:32.470968+0000","last_fullsized":"2024-04-25T11:20:32.470968+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:11.840736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.471363+0000","last_change":"2024-04-25T11:20:32.462963+0000","last_active":"2024-04-25T11:20:32.471363+0000","last_peered":"2024-04-25T11:20:32.471363+0000","last_clean":"2024-04-25T11:20:32.471363+0000","last_became_active":"2024-04-25T11:20:32.462132+0000","last_became_peered":"2024-04-25T11:20:32.462132+0000","last_unstale":"2024-04-25T11:20:32.471363+0000","last_undegraded":"2024-04-25T11:20:32.471363+0000","last_fullsized":"2024-04-25T11:20:32.471363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:03.896712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.434863+0000","last_change":"2024-04-25T11:20:32.430790+0000","last_active":"2024-04-25T11:20:32.434863+0000","last_peered":"2024-04-25T11:20:32.434863+0000","last_clean":"2024-04-25T11:20:32.434863+0000","last_became_active":"2024-04-25T11:20:32.429907+0000","last_became_peered":"2024-04-25T11:20:32.429907+0000","last_unstale":"2024-04-25T11:20:32.434863+0000","last_undegraded":"2024-04-25T11:20:32.434863+0000","last_fullsized":"2024-04-25T11:20:32.434863+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:22.158494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.437926+0000","last_change":"2024-04-25T11:20:32.433782+0000","last_active":"2024-04-25T11:20:32.437926+0000","last_peered":"2024-04-25T11:20:32.437926+0000","last_clean":"2024-04-25T11:20:32.437926+0000","last_became_active":"2024-04-25T11:20:32.433477+0000","last_became_peered":"2024-04-25T11:20:32.433477+0000","last_unstale":"2024-04-25T11:20:32.437926+0000","last_undegraded":"2024-04-25T11:20:32.437926+0000","last_fullsized":"2024-04-25T11:20:32.437926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:21.990942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.470073+0000","last_change":"2024-04-25T11:20:32.454287+0000","last_active":"2024-04-25T11:20:32.470073+0000","last_peered":"2024-04-25T11:20:32.470073+0000","last_clean":"2024-04-25T11:20:32.470073+0000","last_became_active":"2024-04-25T11:20:32.453370+0000","last_became_peered":"2024-04-25T11:20:32.453370+0000","last_unstale":"2024-04-25T11:20:32.470073+0000","last_undegraded":"2024-04-25T11:20:32.470073+0000","last_fullsized":"2024-04-25T11:20:32.470073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:54.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479018+0000","last_change":"2024-04-25T11:20:32.454554+0000","last_active":"2024-04-25T11:20:32.479018+0000","last_peered":"2024-04-25T11:20:32.479018+0000","last_clean":"2024-04-25T11:20:32.479018+0000","last_became_active":"2024-04-25T11:20:32.447829+0000","last_became_peered":"2024-04-25T11:20:32.447829+0000","last_unstale":"2024-04-25T11:20:32.479018+0000","last_undegraded":"2024-04-25T11:20:32.479018+0000","last_fullsized":"2024-04-25T11:20:32.479018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:29.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415752+0000","last_change":"2024-04-25T11:20:32.430914+0000","last_active":"2024-04-25T11:20:33.415752+0000","last_peered":"2024-04-25T11:20:33.415752+0000","last_clean":"2024-04-25T11:20:33.415752+0000","last_became_active":"2024-04-25T11:20:32.430218+0000","last_became_peered":"2024-04-25T11:20:32.430218+0000","last_unstale":"2024-04-25T11:20:33.415752+0000","last_undegraded":"2024-04-25T11:20:33.415752+0000","last_fullsized":"2024-04-25T11:20:33.415752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:51.320902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902359+0000","last_change":"2024-04-25T11:20:32.429534+0000","last_active":"2024-04-25T11:20:33.902359+0000","last_peered":"2024-04-25T11:20:33.902359+0000","last_clean":"2024-04-25T11:20:33.902359+0000","last_became_active":"2024-04-25T11:20:32.428338+0000","last_became_peered":"2024-04-25T11:20:32.428338+0000","last_unstale":"2024-04-25T11:20:33.902359+0000","last_undegraded":"2024-04-25T11:20:33.902359+0000","last_fullsized":"2024-04-25T11:20:33.902359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:09.836157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.482268+0000","last_change":"2024-04-25T11:20:32.472104+0000","last_active":"2024-04-25T11:20:32.482268+0000","last_peered":"2024-04-25T11:20:32.482268+0000","last_clean":"2024-04-25T11:20:32.482268+0000","last_became_active":"2024-04-25T11:20:32.470774+0000","last_became_peered":"2024-04-25T11:20:32.470774+0000","last_unstale":"2024-04-25T11:20:32.482268+0000","last_undegraded":"2024-04-25T11:20:32.482268+0000","last_fullsized":"2024-04-25T11:20:32.482268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:44.004137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.475658+0000","last_change":"2024-04-25T11:20:32.471795+0000","last_active":"2024-04-25T11:20:32.475658+0000","last_peered":"2024-04-25T11:20:32.475658+0000","last_clean":"2024-04-25T11:20:32.475658+0000","last_became_active":"2024-04-25T11:20:32.470430+0000","last_became_peered":"2024-04-25T11:20:32.470430+0000","last_unstale":"2024-04-25T11:20:32.475658+0000","last_undegraded":"2024-04-25T11:20:32.475658+0000","last_fullsized":"2024-04-25T11:20:32.475658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.006165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.480439+0000","last_change":"2024-04-25T11:20:32.476501+0000","last_active":"2024-04-25T11:20:32.480439+0000","last_peered":"2024-04-25T11:20:32.480439+0000","last_clean":"2024-04-25T11:20:32.480439+0000","last_became_active":"2024-04-25T11:20:32.475810+0000","last_became_peered":"2024-04-25T11:20:32.475810+0000","last_unstale":"2024-04-25T11:20:32.480439+0000","last_undegraded":"2024-04-25T11:20:32.480439+0000","last_fullsized":"2024-04-25T11:20:32.480439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:44.004137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902484+0000","last_change":"2024-04-25T11:20:32.436111+0000","last_active":"2024-04-25T11:20:33.902484+0000","last_peered":"2024-04-25T11:20:33.902484+0000","last_clean":"2024-04-25T11:20:33.902484+0000","last_became_active":"2024-04-25T11:20:32.434315+0000","last_became_peered":"2024-04-25T11:20:32.434315+0000","last_unstale":"2024-04-25T11:20:33.902484+0000","last_undegraded":"2024-04-25T11:20:33.902484+0000","last_fullsized":"2024-04-25T11:20:33.902484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.780852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902140+0000","last_change":"2024-04-25T11:20:32.426964+0000","last_active":"2024-04-25T11:20:33.902140+0000","last_peered":"2024-04-25T11:20:33.902140+0000","last_clean":"2024-04-25T11:20:33.902140+0000","last_became_active":"2024-04-25T11:20:32.426586+0000","last_became_peered":"2024-04-25T11:20:32.426586+0000","last_unstale":"2024-04-25T11:20:33.902140+0000","last_undegraded":"2024-04-25T11:20:33.902140+0000","last_fullsized":"2024-04-25T11:20:33.902140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:15.990703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415952+0000","last_change":"2024-04-25T11:20:32.463304+0000","last_active":"2024-04-25T11:20:33.415952+0000","last_peered":"2024-04-25T11:20:33.415952+0000","last_clean":"2024-04-25T11:20:33.415952+0000","last_became_active":"2024-04-25T11:20:32.463001+0000","last_became_peered":"2024-04-25T11:20:32.463001+0000","last_unstale":"2024-04-25T11:20:33.415952+0000","last_undegraded":"2024-04-25T11:20:33.415952+0000","last_fullsized":"2024-04-25T11:20:33.415952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:03.133009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.444633+0000","last_change":"2024-04-25T11:20:32.434745+0000","last_active":"2024-04-25T11:20:32.444633+0000","last_peered":"2024-04-25T11:20:32.444633+0000","last_clean":"2024-04-25T11:20:32.444633+0000","last_became_active":"2024-04-25T11:20:32.429708+0000","last_became_peered":"2024-04-25T11:20:32.429708+0000","last_unstale":"2024-04-25T11:20:32.444633+0000","last_undegraded":"2024-04-25T11:20:32.444633+0000","last_fullsized":"2024-04-25T11:20:32.444633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:54.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.434282+0000","last_change":"2024-04-25T11:20:32.424086+0000","last_active":"2024-04-25T11:20:32.434282+0000","last_peered":"2024-04-25T11:20:32.434282+0000","last_clean":"2024-04-25T11:20:32.434282+0000","last_became_active":"2024-04-25T11:20:32.423314+0000","last_became_peered":"2024-04-25T11:20:32.423314+0000","last_unstale":"2024-04-25T11:20:32.434282+0000","last_undegraded":"2024-04-25T11:20:32.434282+0000","last_fullsized":"2024-04-25T11:20:32.434282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.414723+0000","last_change":"2024-04-25T11:20:32.449227+0000","last_active":"2024-04-25T11:20:33.414723+0000","last_peered":"2024-04-25T11:20:33.414723+0000","last_clean":"2024-04-25T11:20:33.414723+0000","last_became_active":"2024-04-25T11:20:32.448875+0000","last_became_peered":"2024-04-25T11:20:32.448875+0000","last_unstale":"2024-04-25T11:20:33.414723+0000","last_undegraded":"2024-04-25T11:20:33.414723+0000","last_fullsized":"2024-04-25T11:20:33.414723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.006165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.481897+0000","last_change":"2024-04-25T11:20:32.462363+0000","last_active":"2024-04-25T11:20:32.481897+0000","last_peered":"2024-04-25T11:20:32.481897+0000","last_clean":"2024-04-25T11:20:32.481897+0000","last_became_active":"2024-04-25T11:20:32.461569+0000","last_became_peered":"2024-04-25T11:20:32.461569+0000","last_unstale":"2024-04-25T11:20:32.481897+0000","last_undegraded":"2024-04-25T11:20:32.481897+0000","last_fullsized":"2024-04-25T11:20:32.481897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:34.139129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.430637+0000","last_change":"2024-04-25T11:20:32.428856+0000","last_active":"2024-04-25T11:20:32.430637+0000","last_peered":"2024-04-25T11:20:32.430637+0000","last_clean":"2024-04-25T11:20:32.430637+0000","last_became_active":"2024-04-25T11:20:32.428384+0000","last_became_peered":"2024-04-25T11:20:32.428384+0000","last_unstale":"2024-04-25T11:20:32.430637+0000","last_undegraded":"2024-04-25T11:20:32.430637+0000","last_fullsized":"2024-04-25T11:20:32.430637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902890+0000","last_change":"2024-04-25T11:20:32.450266+0000","last_active":"2024-04-25T11:20:33.902890+0000","last_peered":"2024-04-25T11:20:33.902890+0000","last_clean":"2024-04-25T11:20:33.902890+0000","last_became_active":"2024-04-25T11:20:32.449936+0000","last_became_peered":"2024-04-25T11:20:32.449936+0000","last_unstale":"2024-04-25T11:20:33.902890+0000","last_undegraded":"2024-04-25T11:20:33.902890+0000","last_fullsized":"2024-04-25T11:20:33.902890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:43.455079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.438271+0000","last_change":"2024-04-25T11:20:32.436352+0000","last_active":"2024-04-25T11:20:32.438271+0000","last_peered":"2024-04-25T11:20:32.438271+0000","last_clean":"2024-04-25T11:20:32.438271+0000","last_became_active":"2024-04-25T11:20:32.436034+0000","last_became_peered":"2024-04-25T11:20:32.436034+0000","last_unstale":"2024-04-25T11:20:32.438271+0000","last_undegraded":"2024-04-25T11:20:32.438271+0000","last_fullsized":"2024-04-25T11:20:32.438271+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:54.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479062+0000","last_change":"2024-04-25T11:20:32.455350+0000","last_active":"2024-04-25T11:20:32.479062+0000","last_peered":"2024-04-25T11:20:32.479062+0000","last_clean":"2024-04-25T11:20:32.479062+0000","last_became_active":"2024-04-25T11:20:32.454456+0000","last_became_peered":"2024-04-25T11:20:32.454456+0000","last_unstale":"2024-04-25T11:20:32.479062+0000","last_undegraded":"2024-04-25T11:20:32.479062+0000","last_fullsized":"2024-04-25T11:20:32.479062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:11.840736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.432238+0000","last_change":"2024-04-25T11:20:32.423941+0000","last_active":"2024-04-25T11:20:32.432238+0000","last_peered":"2024-04-25T11:20:32.432238+0000","last_clean":"2024-04-25T11:20:32.432238+0000","last_became_active":"2024-04-25T11:20:32.423154+0000","last_became_peered":"2024-04-25T11:20:32.423154+0000","last_unstale":"2024-04-25T11:20:32.432238+0000","last_undegraded":"2024-04-25T11:20:32.432238+0000","last_fullsized":"2024-04-25T11:20:32.432238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:18.525729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.470230+0000","last_change":"2024-04-25T11:20:32.463065+0000","last_active":"2024-04-25T11:20:32.470230+0000","last_peered":"2024-04-25T11:20:32.470230+0000","last_clean":"2024-04-25T11:20:32.470230+0000","last_became_active":"2024-04-25T11:20:32.462335+0000","last_became_peered":"2024-04-25T11:20:32.462335+0000","last_unstale":"2024-04-25T11:20:32.470230+0000","last_undegraded":"2024-04-25T11:20:32.470230+0000","last_fullsized":"2024-04-25T11:20:32.470230+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.473547+0000","last_change":"2024-04-25T11:20:32.466349+0000","last_active":"2024-04-25T11:20:32.473547+0000","last_peered":"2024-04-25T11:20:32.473547+0000","last_clean":"2024-04-25T11:20:32.473547+0000","last_became_active":"2024-04-25T11:20:32.465576+0000","last_became_peered":"2024-04-25T11:20:32.465576+0000","last_unstale":"2024-04-25T11:20:32.473547+0000","last_undegraded":"2024-04-25T11:20:32.473547+0000","last_fullsized":"2024-04-25T11:20:32.473547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:30.601537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479486+0000","last_change":"2024-04-25T11:20:32.473437+0000","last_active":"2024-04-25T11:20:32.479486+0000","last_peered":"2024-04-25T11:20:32.479486+0000","last_clean":"2024-04-25T11:20:32.479486+0000","last_became_active":"2024-04-25T11:20:32.472793+0000","last_became_peered":"2024-04-25T11:20:32.472793+0000","last_unstale":"2024-04-25T11:20:32.479486+0000","last_undegraded":"2024-04-25T11:20:32.479486+0000","last_fullsized":"2024-04-25T11:20:32.479486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:56.022841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413561+0000","last_change":"2024-04-25T11:20:32.428037+0000","last_active":"2024-04-25T11:20:33.413561+0000","last_peered":"2024-04-25T11:20:33.413561+0000","last_clean":"2024-04-25T11:20:33.413561+0000","last_became_active":"2024-04-25T11:20:32.427628+0000","last_became_peered":"2024-04-25T11:20:32.427628+0000","last_unstale":"2024-04-25T11:20:33.413561+0000","last_undegraded":"2024-04-25T11:20:33.413561+0000","last_fullsized":"2024-04-25T11:20:33.413561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:29.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.482272+0000","last_change":"2024-04-25T11:20:31.455421+0000","last_active":"2024-04-25T11:20:32.482272+0000","last_peered":"2024-04-25T11:20:32.482272+0000","last_clean":"2024-04-25T11:20:32.482272+0000","last_became_active":"2024-04-25T11:20:31.454529+0000","last_became_peered":"2024-04-25T11:20:31.454529+0000","last_unstale":"2024-04-25T11:20:32.482272+0000","last_undegraded":"2024-04-25T11:20:32.482272+0000","last_fullsized":"2024-04-25T11:20:32.482272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:57.474821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.477078+0000","last_change":"2024-04-25T11:20:32.473893+0000","last_active":"2024-04-25T11:20:32.477078+0000","last_peered":"2024-04-25T11:20:32.477078+0000","last_clean":"2024-04-25T11:20:32.477078+0000","last_became_active":"2024-04-25T11:20:32.473178+0000","last_became_peered":"2024-04-25T11:20:32.473178+0000","last_unstale":"2024-04-25T11:20:32.477078+0000","last_undegraded":"2024-04-25T11:20:32.477078+0000","last_fullsized":"2024-04-25T11:20:32.477078+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:18.525729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413872+0000","last_change":"2024-04-25T11:20:31.443288+0000","last_active":"2024-04-25T11:20:33.413872+0000","last_peered":"2024-04-25T11:20:33.413872+0000","last_clean":"2024-04-25T11:20:33.413872+0000","last_became_active":"2024-04-25T11:20:31.442848+0000","last_became_peered":"2024-04-25T11:20:31.442848+0000","last_unstale":"2024-04-25T11:20:33.413872+0000","last_undegraded":"2024-04-25T11:20:33.413872+0000","last_fullsized":"2024-04-25T11:20:33.413872+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:50.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.436818+0000","last_change":"2024-04-25T11:20:32.432822+0000","last_active":"2024-04-25T11:20:32.436818+0000","last_peered":"2024-04-25T11:20:32.436818+0000","last_clean":"2024-04-25T11:20:32.436818+0000","last_became_active":"2024-04-25T11:20:32.431863+0000","last_became_peered":"2024-04-25T11:20:32.431863+0000","last_unstale":"2024-04-25T11:20:32.436818+0000","last_undegraded":"2024-04-25T11:20:32.436818+0000","last_fullsized":"2024-04-25T11:20:32.436818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:21.990942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902240+0000","last_change":"2024-04-25T11:20:31.426377+0000","last_active":"2024-04-25T11:20:33.902240+0000","last_peered":"2024-04-25T11:20:33.902240+0000","last_clean":"2024-04-25T11:20:33.902240+0000","last_became_active":"2024-04-25T11:20:31.425729+0000","last_became_peered":"2024-04-25T11:20:31.425729+0000","last_unstale":"2024-04-25T11:20:33.902240+0000","last_undegraded":"2024-04-25T11:20:33.902240+0000","last_fullsized":"2024-04-25T11:20:33.902240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:02.886342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.438299+0000","last_change":"2024-04-25T11:20:32.436365+0000","last_active":"2024-04-25T11:20:32.438299+0000","last_peered":"2024-04-25T11:20:32.438299+0000","last_clean":"2024-04-25T11:20:32.438299+0000","last_became_active":"2024-04-25T11:20:32.436046+0000","last_became_peered":"2024-04-25T11:20:32.436046+0000","last_unstale":"2024-04-25T11:20:32.438299+0000","last_undegraded":"2024-04-25T11:20:32.438299+0000","last_fullsized":"2024-04-25T11:20:32.438299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:03.896712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.416263+0000","last_change":"2024-04-25T11:20:31.426977+0000","last_active":"2024-04-25T11:20:33.416263+0000","last_peered":"2024-04-25T11:20:33.416263+0000","last_clean":"2024-04-25T11:20:33.416263+0000","last_became_active":"2024-04-25T11:20:31.426593+0000","last_became_peered":"2024-04-25T11:20:31.426593+0000","last_unstale":"2024-04-25T11:20:33.416263+0000","last_undegraded":"2024-04-25T11:20:33.416263+0000","last_fullsized":"2024-04-25T11:20:33.416263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:45.728761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479392+0000","last_change":"2024-04-25T11:20:32.473697+0000","last_active":"2024-04-25T11:20:32.479392+0000","last_peered":"2024-04-25T11:20:32.479392+0000","last_clean":"2024-04-25T11:20:32.479392+0000","last_became_active":"2024-04-25T11:20:32.472986+0000","last_became_peered":"2024-04-25T11:20:32.472986+0000","last_unstale":"2024-04-25T11:20:32.479392+0000","last_undegraded":"2024-04-25T11:20:32.479392+0000","last_fullsized":"2024-04-25T11:20:32.479392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.215661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.472547+0000","last_change":"2024-04-25T11:20:31.448054+0000","last_active":"2024-04-25T11:20:32.472547+0000","last_peered":"2024-04-25T11:20:32.472547+0000","last_clean":"2024-04-25T11:20:32.472547+0000","last_became_active":"2024-04-25T11:20:31.447194+0000","last_became_peered":"2024-04-25T11:20:31.447194+0000","last_unstale":"2024-04-25T11:20:32.472547+0000","last_undegraded":"2024-04-25T11:20:32.472547+0000","last_fullsized":"2024-04-25T11:20:32.472547+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:02.122639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.477951+0000","last_change":"2024-04-25T11:20:32.452317+0000","last_active":"2024-04-25T11:20:32.477951+0000","last_peered":"2024-04-25T11:20:32.477951+0000","last_clean":"2024-04-25T11:20:32.477951+0000","last_became_active":"2024-04-25T11:20:32.451578+0000","last_became_peered":"2024-04-25T11:20:32.451578+0000","last_unstale":"2024-04-25T11:20:32.477951+0000","last_undegraded":"2024-04-25T11:20:32.477951+0000","last_fullsized":"2024-04-25T11:20:32.477951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.471579+0000","last_change":"2024-04-25T11:20:31.445827+0000","last_active":"2024-04-25T11:20:32.471579+0000","last_peered":"2024-04-25T11:20:32.471579+0000","last_clean":"2024-04-25T11:20:32.471579+0000","last_became_active":"2024-04-25T11:20:31.444888+0000","last_became_peered":"2024-04-25T11:20:31.444888+0000","last_unstale":"2024-04-25T11:20:32.471579+0000","last_undegraded":"2024-04-25T11:20:32.471579+0000","last_fullsized":"2024-04-25T11:20:32.471579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:19.938517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.471484+0000","last_change":"2024-04-25T11:20:32.445482+0000","last_active":"2024-04-25T11:20:32.471484+0000","last_peered":"2024-04-25T11:20:32.471484+0000","last_clean":"2024-04-25T11:20:32.471484+0000","last_became_active":"2024-04-25T11:20:32.444461+0000","last_became_peered":"2024-04-25T11:20:32.444461+0000","last_unstale":"2024-04-25T11:20:32.471484+0000","last_undegraded":"2024-04-25T11:20:32.471484+0000","last_fullsized":"2024-04-25T11:20:32.471484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:03.896712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901424+0000","last_change":"2024-04-25T11:20:31.426671+0000","last_active":"2024-04-25T11:20:33.901424+0000","last_peered":"2024-04-25T11:20:33.901424+0000","last_clean":"2024-04-25T11:20:33.901424+0000","last_became_active":"2024-04-25T11:20:31.425832+0000","last_became_peered":"2024-04-25T11:20:31.425832+0000","last_unstale":"2024-04-25T11:20:33.901424+0000","last_undegraded":"2024-04-25T11:20:33.901424+0000","last_fullsized":"2024-04-25T11:20:33.901424+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:47.815189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.448685+0000","last_change":"2024-04-25T11:20:32.446484+0000","last_active":"2024-04-25T11:20:32.448685+0000","last_peered":"2024-04-25T11:20:32.448685+0000","last_clean":"2024-04-25T11:20:32.448685+0000","last_became_active":"2024-04-25T11:20:32.445645+0000","last_became_peered":"2024-04-25T11:20:32.445645+0000","last_unstale":"2024-04-25T11:20:32.448685+0000","last_undegraded":"2024-04-25T11:20:32.448685+0000","last_fullsized":"2024-04-25T11:20:32.448685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:11.840736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.415243+0000","last_change":"2024-04-25T11:20:31.436083+0000","last_active":"2024-04-25T11:20:32.415243+0000","last_peered":"2024-04-25T11:20:32.415243+0000","last_clean":"2024-04-25T11:20:32.415243+0000","last_became_active":"2024-04-25T11:20:31.435270+0000","last_became_peered":"2024-04-25T11:20:31.435270+0000","last_unstale":"2024-04-25T11:20:32.415243+0000","last_undegraded":"2024-04-25T11:20:32.415243+0000","last_fullsized":"2024-04-25T11:20:32.415243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:29.371839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.454737+0000","last_change":"2024-04-25T11:20:32.443409+0000","last_active":"2024-04-25T11:20:32.454737+0000","last_peered":"2024-04-25T11:20:32.454737+0000","last_clean":"2024-04-25T11:20:32.454737+0000","last_became_active":"2024-04-25T11:20:32.442118+0000","last_became_peered":"2024-04-25T11:20:32.442118+0000","last_unstale":"2024-04-25T11:20:32.454737+0000","last_undegraded":"2024-04-25T11:20:32.454737+0000","last_fullsized":"2024-04-25T11:20:32.454737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:03.896712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902695+0000","last_change":"2024-04-25T11:20:31.445363+0000","last_active":"2024-04-25T11:20:33.902695+0000","last_peered":"2024-04-25T11:20:33.902695+0000","last_clean":"2024-04-25T11:20:33.902695+0000","last_became_active":"2024-04-25T11:20:31.445037+0000","last_became_peered":"2024-04-25T11:20:31.445037+0000","last_unstale":"2024-04-25T11:20:33.902695+0000","last_undegraded":"2024-04-25T11:20:33.902695+0000","last_fullsized":"2024-04-25T11:20:33.902695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:28.640067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902715+0000","last_change":"2024-04-25T11:20:32.434340+0000","last_active":"2024-04-25T11:20:33.902715+0000","last_peered":"2024-04-25T11:20:33.902715+0000","last_clean":"2024-04-25T11:20:33.902715+0000","last_became_active":"2024-04-25T11:20:32.433962+0000","last_became_peered":"2024-04-25T11:20:32.433962+0000","last_unstale":"2024-04-25T11:20:33.902715+0000","last_undegraded":"2024-04-25T11:20:33.902715+0000","last_fullsized":"2024-04-25T11:20:33.902715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:35.636848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.414145+0000","last_change":"2024-04-25T11:20:31.424396+0000","last_active":"2024-04-25T11:20:33.414145+0000","last_peered":"2024-04-25T11:20:33.414145+0000","last_clean":"2024-04-25T11:20:33.414145+0000","last_became_active":"2024-04-25T11:20:31.423961+0000","last_became_peered":"2024-04-25T11:20:31.423961+0000","last_unstale":"2024-04-25T11:20:33.414145+0000","last_undegraded":"2024-04-25T11:20:33.414145+0000","last_fullsized":"2024-04-25T11:20:33.414145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:02.122639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902890+0000","last_change":"2024-04-25T11:20:32.453941+0000","last_active":"2024-04-25T11:20:33.902890+0000","last_peered":"2024-04-25T11:20:33.902890+0000","last_clean":"2024-04-25T11:20:33.902890+0000","last_became_active":"2024-04-25T11:20:32.453673+0000","last_became_peered":"2024-04-25T11:20:32.453673+0000","last_unstale":"2024-04-25T11:20:33.902890+0000","last_undegraded":"2024-04-25T11:20:33.902890+0000","last_fullsized":"2024-04-25T11:20:33.902890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:15.873495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.482526+0000","last_change":"2024-04-25T11:20:31.449797+0000","last_active":"2024-04-25T11:20:32.482526+0000","last_peered":"2024-04-25T11:20:32.482526+0000","last_clean":"2024-04-25T11:20:32.482526+0000","last_became_active":"2024-04-25T11:20:31.448637+0000","last_became_peered":"2024-04-25T11:20:31.448637+0000","last_unstale":"2024-04-25T11:20:32.482526+0000","last_undegraded":"2024-04-25T11:20:32.482526+0000","last_fullsized":"2024-04-25T11:20:32.482526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:53.828515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.481749+0000","last_change":"2024-04-25T11:20:32.460594+0000","last_active":"2024-04-25T11:20:32.481749+0000","last_peered":"2024-04-25T11:20:32.481749+0000","last_clean":"2024-04-25T11:20:32.481749+0000","last_became_active":"2024-04-25T11:20:32.457327+0000","last_became_peered":"2024-04-25T11:20:32.457327+0000","last_unstale":"2024-04-25T11:20:32.481749+0000","last_undegraded":"2024-04-25T11:20:32.481749+0000","last_fullsized":"2024-04-25T11:20:32.481749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.215661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902841+0000","last_change":"2024-04-25T11:20:31.427092+0000","last_active":"2024-04-25T11:20:33.902841+0000","last_peered":"2024-04-25T11:20:33.902841+0000","last_clean":"2024-04-25T11:20:33.902841+0000","last_became_active":"2024-04-25T11:20:31.426396+0000","last_became_peered":"2024-04-25T11:20:31.426396+0000","last_unstale":"2024-04-25T11:20:33.902841+0000","last_undegraded":"2024-04-25T11:20:33.902841+0000","last_fullsized":"2024-04-25T11:20:33.902841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:33.759006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.449056+0000","last_change":"2024-04-25T11:20:32.446126+0000","last_active":"2024-04-25T11:20:32.449056+0000","last_peered":"2024-04-25T11:20:32.449056+0000","last_clean":"2024-04-25T11:20:32.449056+0000","last_became_active":"2024-04-25T11:20:32.445725+0000","last_became_peered":"2024-04-25T11:20:32.445725+0000","last_unstale":"2024-04-25T11:20:32.449056+0000","last_undegraded":"2024-04-25T11:20:32.449056+0000","last_fullsized":"2024-04-25T11:20:32.449056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:29.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.472440+0000","last_change":"2024-04-25T11:20:31.446205+0000","last_active":"2024-04-25T11:20:32.472440+0000","last_peered":"2024-04-25T11:20:32.472440+0000","last_clean":"2024-04-25T11:20:32.472440+0000","last_became_active":"2024-04-25T11:20:31.443131+0000","last_became_peered":"2024-04-25T11:20:31.443131+0000","last_unstale":"2024-04-25T11:20:32.472440+0000","last_undegraded":"2024-04-25T11:20:32.472440+0000","last_fullsized":"2024-04-25T11:20:32.472440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:29.591167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902491+0000","last_change":"2024-04-25T11:20:32.425087+0000","last_active":"2024-04-25T11:20:33.902491+0000","last_peered":"2024-04-25T11:20:33.902491+0000","last_clean":"2024-04-25T11:20:33.902491+0000","last_became_active":"2024-04-25T11:20:32.424703+0000","last_became_peered":"2024-04-25T11:20:32.424703+0000","last_unstale":"2024-04-25T11:20:33.902491+0000","last_undegraded":"2024-04-25T11:20:33.902491+0000","last_fullsized":"2024-04-25T11:20:33.902491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:23.526725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.423168+0000","last_change":"2024-04-25T11:20:31.435320+0000","last_active":"2024-04-25T11:20:32.423168+0000","last_peered":"2024-04-25T11:20:32.423168+0000","last_clean":"2024-04-25T11:20:32.423168+0000","last_became_active":"2024-04-25T11:20:31.434942+0000","last_became_peered":"2024-04-25T11:20:31.434942+0000","last_unstale":"2024-04-25T11:20:32.423168+0000","last_undegraded":"2024-04-25T11:20:32.423168+0000","last_fullsized":"2024-04-25T11:20:32.423168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:57.474821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.480543+0000","last_change":"2024-04-25T11:20:32.477985+0000","last_active":"2024-04-25T11:20:32.480543+0000","last_peered":"2024-04-25T11:20:32.480543+0000","last_clean":"2024-04-25T11:20:32.480543+0000","last_became_active":"2024-04-25T11:20:32.477365+0000","last_became_peered":"2024-04-25T11:20:32.477365+0000","last_unstale":"2024-04-25T11:20:32.480543+0000","last_undegraded":"2024-04-25T11:20:32.480543+0000","last_fullsized":"2024-04-25T11:20:32.480543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:54.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.902464+0000","last_change":"2024-04-25T11:20:31.445398+0000","last_active":"2024-04-25T11:20:33.902464+0000","last_peered":"2024-04-25T11:20:33.902464+0000","last_clean":"2024-04-25T11:20:33.902464+0000","last_became_active":"2024-04-25T11:20:31.445020+0000","last_became_peered":"2024-04-25T11:20:31.445020+0000","last_unstale":"2024-04-25T11:20:33.902464+0000","last_undegraded":"2024-04-25T11:20:33.902464+0000","last_fullsized":"2024-04-25T11:20:33.902464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:53.828515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.478539+0000","last_change":"2024-04-25T11:20:32.475810+0000","last_active":"2024-04-25T11:20:32.478539+0000","last_peered":"2024-04-25T11:20:32.478539+0000","last_clean":"2024-04-25T11:20:32.478539+0000","last_became_active":"2024-04-25T11:20:32.474934+0000","last_became_peered":"2024-04-25T11:20:32.474934+0000","last_unstale":"2024-04-25T11:20:32.478539+0000","last_undegraded":"2024-04-25T11:20:32.478539+0000","last_fullsized":"2024-04-25T11:20:32.478539+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901566+0000","last_change":"2024-04-25T11:20:31.429431+0000","last_active":"2024-04-25T11:20:33.901566+0000","last_peered":"2024-04-25T11:20:33.901566+0000","last_clean":"2024-04-25T11:20:33.901566+0000","last_became_active":"2024-04-25T11:20:31.428993+0000","last_became_peered":"2024-04-25T11:20:31.428993+0000","last_unstale":"2024-04-25T11:20:33.901566+0000","last_undegraded":"2024-04-25T11:20:33.901566+0000","last_fullsized":"2024-04-25T11:20:33.901566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:21.148124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.471932+0000","last_change":"2024-04-25T11:20:32.455223+0000","last_active":"2024-04-25T11:20:32.471932+0000","last_peered":"2024-04-25T11:20:32.471932+0000","last_clean":"2024-04-25T11:20:32.471932+0000","last_became_active":"2024-04-25T11:20:32.454296+0000","last_became_peered":"2024-04-25T11:20:32.454296+0000","last_unstale":"2024-04-25T11:20:32.471932+0000","last_undegraded":"2024-04-25T11:20:32.471932+0000","last_fullsized":"2024-04-25T11:20:32.471932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.440305+0000","last_change":"2024-04-25T11:20:31.429207+0000","last_active":"2024-04-25T11:20:32.440305+0000","last_peered":"2024-04-25T11:20:32.440305+0000","last_clean":"2024-04-25T11:20:32.440305+0000","last_became_active":"2024-04-25T11:20:31.428781+0000","last_became_peered":"2024-04-25T11:20:31.428781+0000","last_unstale":"2024-04-25T11:20:32.440305+0000","last_undegraded":"2024-04-25T11:20:32.440305+0000","last_fullsized":"2024-04-25T11:20:32.440305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:29.591167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.482320+0000","last_change":"2024-04-25T11:20:32.474150+0000","last_active":"2024-04-25T11:20:32.482320+0000","last_peered":"2024-04-25T11:20:32.482320+0000","last_clean":"2024-04-25T11:20:32.482320+0000","last_became_active":"2024-04-25T11:20:32.473578+0000","last_became_peered":"2024-04-25T11:20:32.473578+0000","last_unstale":"2024-04-25T11:20:32.482320+0000","last_undegraded":"2024-04-25T11:20:32.482320+0000","last_fullsized":"2024-04-25T11:20:32.482320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:54.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.469254+0000","last_change":"2024-04-25T11:20:31.445112+0000","last_active":"2024-04-25T11:20:32.469254+0000","last_peered":"2024-04-25T11:20:32.469254+0000","last_clean":"2024-04-25T11:20:32.469254+0000","last_became_active":"2024-04-25T11:20:31.444214+0000","last_became_peered":"2024-04-25T11:20:31.444214+0000","last_unstale":"2024-04-25T11:20:32.469254+0000","last_undegraded":"2024-04-25T11:20:32.469254+0000","last_fullsized":"2024-04-25T11:20:32.469254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:18.138532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.438338+0000","last_change":"2024-04-25T11:20:32.432742+0000","last_active":"2024-04-25T11:20:32.438338+0000","last_peered":"2024-04-25T11:20:32.438338+0000","last_clean":"2024-04-25T11:20:32.438338+0000","last_became_active":"2024-04-25T11:20:32.432297+0000","last_became_peered":"2024-04-25T11:20:32.432297+0000","last_unstale":"2024-04-25T11:20:32.438338+0000","last_undegraded":"2024-04-25T11:20:32.438338+0000","last_fullsized":"2024-04-25T11:20:32.438338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:22.158494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.472101+0000","last_change":"2024-04-25T11:20:31.444332+0000","last_active":"2024-04-25T11:20:32.472101+0000","last_peered":"2024-04-25T11:20:32.472101+0000","last_clean":"2024-04-25T11:20:32.472101+0000","last_became_active":"2024-04-25T11:20:31.443377+0000","last_became_peered":"2024-04-25T11:20:31.443377+0000","last_unstale":"2024-04-25T11:20:32.472101+0000","last_undegraded":"2024-04-25T11:20:32.472101+0000","last_fullsized":"2024-04-25T11:20:32.472101+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:57.474821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415974+0000","last_change":"2024-04-25T11:20:32.450092+0000","last_active":"2024-04-25T11:20:33.415974+0000","last_peered":"2024-04-25T11:20:33.415974+0000","last_clean":"2024-04-25T11:20:33.415974+0000","last_became_active":"2024-04-25T11:20:32.449702+0000","last_became_peered":"2024-04-25T11:20:32.449702+0000","last_unstale":"2024-04-25T11:20:33.415974+0000","last_undegraded":"2024-04-25T11:20:33.415974+0000","last_fullsized":"2024-04-25T11:20:33.415974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:18.525729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.416246+0000","last_change":"2024-04-25T11:20:31.431649+0000","last_active":"2024-04-25T11:20:32.416246+0000","last_peered":"2024-04-25T11:20:32.416246+0000","last_clean":"2024-04-25T11:20:32.416246+0000","last_became_active":"2024-04-25T11:20:31.430754+0000","last_became_peered":"2024-04-25T11:20:31.430754+0000","last_unstale":"2024-04-25T11:20:32.416246+0000","last_undegraded":"2024-04-25T11:20:32.416246+0000","last_fullsized":"2024-04-25T11:20:32.416246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:44.072745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.416184+0000","last_change":"2024-04-25T11:20:32.429410+0000","last_active":"2024-04-25T11:20:33.416184+0000","last_peered":"2024-04-25T11:20:33.416184+0000","last_clean":"2024-04-25T11:20:33.416184+0000","last_became_active":"2024-04-25T11:20:32.429082+0000","last_became_peered":"2024-04-25T11:20:32.429082+0000","last_unstale":"2024-04-25T11:20:33.416184+0000","last_undegraded":"2024-04-25T11:20:33.416184+0000","last_fullsized":"2024-04-25T11:20:33.416184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413920+0000","last_change":"2024-04-25T11:20:31.421932+0000","last_active":"2024-04-25T11:20:33.413920+0000","last_peered":"2024-04-25T11:20:33.413920+0000","last_clean":"2024-04-25T11:20:33.413920+0000","last_became_active":"2024-04-25T11:20:31.421590+0000","last_became_peered":"2024-04-25T11:20:31.421590+0000","last_unstale":"2024-04-25T11:20:33.413920+0000","last_undegraded":"2024-04-25T11:20:33.413920+0000","last_fullsized":"2024-04-25T11:20:33.413920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:57.474821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415644+0000","last_change":"2024-04-25T11:20:32.431199+0000","last_active":"2024-04-25T11:20:33.415644+0000","last_peered":"2024-04-25T11:20:33.415644+0000","last_clean":"2024-04-25T11:20:33.415644+0000","last_became_active":"2024-04-25T11:20:32.430607+0000","last_became_peered":"2024-04-25T11:20:32.430607+0000","last_unstale":"2024-04-25T11:20:33.415644+0000","last_undegraded":"2024-04-25T11:20:33.415644+0000","last_fullsized":"2024-04-25T11:20:33.415644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:58.485191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.438137+0000","last_change":"2024-04-25T11:20:31.442153+0000","last_active":"2024-04-25T11:20:32.438137+0000","last_peered":"2024-04-25T11:20:32.438137+0000","last_clean":"2024-04-25T11:20:32.438137+0000","last_became_active":"2024-04-25T11:20:31.441192+0000","last_became_peered":"2024-04-25T11:20:31.441192+0000","last_unstale":"2024-04-25T11:20:32.438137+0000","last_undegraded":"2024-04-25T11:20:32.438137+0000","last_fullsized":"2024-04-25T11:20:32.438137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:18.138532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415227+0000","last_change":"2024-04-25T11:20:32.427267+0000","last_active":"2024-04-25T11:20:33.415227+0000","last_peered":"2024-04-25T11:20:33.415227+0000","last_clean":"2024-04-25T11:20:33.415227+0000","last_became_active":"2024-04-25T11:20:32.426961+0000","last_became_peered":"2024-04-25T11:20:32.426961+0000","last_unstale":"2024-04-25T11:20:33.415227+0000","last_undegraded":"2024-04-25T11:20:33.415227+0000","last_fullsized":"2024-04-25T11:20:33.415227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:45.083115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.415921+0000","last_change":"2024-04-25T11:20:31.445706+0000","last_active":"2024-04-25T11:20:32.415921+0000","last_peered":"2024-04-25T11:20:32.415921+0000","last_clean":"2024-04-25T11:20:32.415921+0000","last_became_active":"2024-04-25T11:20:31.443158+0000","last_became_peered":"2024-04-25T11:20:31.443158+0000","last_unstale":"2024-04-25T11:20:32.415921+0000","last_undegraded":"2024-04-25T11:20:32.415921+0000","last_fullsized":"2024-04-25T11:20:32.415921+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:21.456266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.479409+0000","last_change":"2024-04-25T11:20:32.473907+0000","last_active":"2024-04-25T11:20:32.479409+0000","last_peered":"2024-04-25T11:20:32.479409+0000","last_clean":"2024-04-25T11:20:32.479409+0000","last_became_active":"2024-04-25T11:20:32.472640+0000","last_became_peered":"2024-04-25T11:20:32.472640+0000","last_unstale":"2024-04-25T11:20:32.479409+0000","last_undegraded":"2024-04-25T11:20:32.479409+0000","last_fullsized":"2024-04-25T11:20:32.479409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:34.139129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.436706+0000","last_change":"2024-04-25T11:20:31.454488+0000","last_active":"2024-04-25T11:20:32.436706+0000","last_peered":"2024-04-25T11:20:32.436706+0000","last_clean":"2024-04-25T11:20:32.436706+0000","last_became_active":"2024-04-25T11:20:31.453689+0000","last_became_peered":"2024-04-25T11:20:31.453689+0000","last_unstale":"2024-04-25T11:20:32.436706+0000","last_undegraded":"2024-04-25T11:20:32.436706+0000","last_fullsized":"2024-04-25T11:20:32.436706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:20.980572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.415969+0000","last_change":"2024-04-25T11:20:32.463198+0000","last_active":"2024-04-25T11:20:33.415969+0000","last_peered":"2024-04-25T11:20:33.415969+0000","last_clean":"2024-04-25T11:20:33.415969+0000","last_became_active":"2024-04-25T11:20:32.462875+0000","last_became_peered":"2024-04-25T11:20:32.462875+0000","last_unstale":"2024-04-25T11:20:33.415969+0000","last_undegraded":"2024-04-25T11:20:33.415969+0000","last_fullsized":"2024-04-25T11:20:33.415969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:30.601537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.478184+0000","last_change":"2024-04-25T11:20:31.452982+0000","last_active":"2024-04-25T11:20:32.478184+0000","last_peered":"2024-04-25T11:20:32.478184+0000","last_clean":"2024-04-25T11:20:32.478184+0000","last_became_active":"2024-04-25T11:20:31.452614+0000","last_became_peered":"2024-04-25T11:20:31.452614+0000","last_unstale":"2024-04-25T11:20:32.478184+0000","last_undegraded":"2024-04-25T11:20:32.478184+0000","last_fullsized":"2024-04-25T11:20:32.478184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:17.515359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.478764+0000","last_change":"2024-04-25T11:20:32.456269+0000","last_active":"2024-04-25T11:20:32.478764+0000","last_peered":"2024-04-25T11:20:32.478764+0000","last_clean":"2024-04-25T11:20:32.478764+0000","last_became_active":"2024-04-25T11:20:32.455367+0000","last_became_peered":"2024-04-25T11:20:32.455367+0000","last_unstale":"2024-04-25T11:20:32.478764+0000","last_undegraded":"2024-04-25T11:20:32.478764+0000","last_fullsized":"2024-04-25T11:20:32.478764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.006165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.414431+0000","last_change":"2024-04-25T11:20:31.418753+0000","last_active":"2024-04-25T11:20:33.414431+0000","last_peered":"2024-04-25T11:20:33.414431+0000","last_clean":"2024-04-25T11:20:33.414431+0000","last_became_active":"2024-04-25T11:20:31.418381+0000","last_became_peered":"2024-04-25T11:20:31.418381+0000","last_unstale":"2024-04-25T11:20:33.414431+0000","last_undegraded":"2024-04-25T11:20:33.414431+0000","last_fullsized":"2024-04-25T11:20:33.414431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:17.515359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413308+0000","last_change":"2024-04-25T11:20:32.427144+0000","last_active":"2024-04-25T11:20:33.413308+0000","last_peered":"2024-04-25T11:20:33.413308+0000","last_clean":"2024-04-25T11:20:33.413308+0000","last_became_active":"2024-04-25T11:20:32.426765+0000","last_became_peered":"2024-04-25T11:20:32.426765+0000","last_unstale":"2024-04-25T11:20:33.413308+0000","last_undegraded":"2024-04-25T11:20:33.413308+0000","last_fullsized":"2024-04-25T11:20:33.413308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.769376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.423780+0000","last_change":"2024-04-25T11:20:31.422133+0000","last_active":"2024-04-25T11:20:32.423780+0000","last_peered":"2024-04-25T11:20:32.423780+0000","last_clean":"2024-04-25T11:20:32.423780+0000","last_became_active":"2024-04-25T11:20:31.421558+0000","last_became_peered":"2024-04-25T11:20:31.421558+0000","last_unstale":"2024-04-25T11:20:32.423780+0000","last_undegraded":"2024-04-25T11:20:32.423780+0000","last_fullsized":"2024-04-25T11:20:32.423780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:02.122639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.467364+0000","last_change":"2024-04-25T11:20:32.465303+0000","last_active":"2024-04-25T11:20:32.467364+0000","last_peered":"2024-04-25T11:20:32.467364+0000","last_clean":"2024-04-25T11:20:32.467364+0000","last_became_active":"2024-04-25T11:20:32.464414+0000","last_became_peered":"2024-04-25T11:20:32.464414+0000","last_unstale":"2024-04-25T11:20:32.467364+0000","last_undegraded":"2024-04-25T11:20:32.467364+0000","last_fullsized":"2024-04-25T11:20:32.467364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:51.320902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.414175+0000","last_change":"2024-04-25T11:20:31.439237+0000","last_active":"2024-04-25T11:20:33.414175+0000","last_peered":"2024-04-25T11:20:33.414175+0000","last_clean":"2024-04-25T11:20:33.414175+0000","last_became_active":"2024-04-25T11:20:31.438862+0000","last_became_peered":"2024-04-25T11:20:31.438862+0000","last_unstale":"2024-04-25T11:20:33.414175+0000","last_undegraded":"2024-04-25T11:20:33.414175+0000","last_fullsized":"2024-04-25T11:20:33.414175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:29.591167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.413758+0000","last_change":"2024-04-25T11:20:32.438678+0000","last_active":"2024-04-25T11:20:33.413758+0000","last_peered":"2024-04-25T11:20:33.413758+0000","last_clean":"2024-04-25T11:20:33.413758+0000","last_became_active":"2024-04-25T11:20:32.438342+0000","last_became_peered":"2024-04-25T11:20:32.438342+0000","last_unstale":"2024-04-25T11:20:33.413758+0000","last_undegraded":"2024-04-25T11:20:33.413758+0000","last_fullsized":"2024-04-25T11:20:33.413758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:11.840736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.423372+0000","last_change":"2024-04-25T11:20:31.438307+0000","last_active":"2024-04-25T11:20:32.423372+0000","last_peered":"2024-04-25T11:20:32.423372+0000","last_clean":"2024-04-25T11:20:32.423372+0000","last_became_active":"2024-04-25T11:20:31.437983+0000","last_became_peered":"2024-04-25T11:20:31.437983+0000","last_unstale":"2024-04-25T11:20:32.423372+0000","last_undegraded":"2024-04-25T11:20:32.423372+0000","last_fullsized":"2024-04-25T11:20:32.423372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:50.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901484+0000","last_change":"2024-04-25T11:20:32.427095+0000","last_active":"2024-04-25T11:20:33.901484+0000","last_peered":"2024-04-25T11:20:33.901484+0000","last_clean":"2024-04-25T11:20:33.901484+0000","last_became_active":"2024-04-25T11:20:32.426584+0000","last_became_peered":"2024-04-25T11:20:32.426584+0000","last_unstale":"2024-04-25T11:20:33.901484+0000","last_undegraded":"2024-04-25T11:20:33.901484+0000","last_fullsized":"2024-04-25T11:20:33.901484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:10.846516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.415957+0000","last_change":"2024-04-25T11:20:31.454753+0000","last_active":"2024-04-25T11:20:32.415957+0000","last_peered":"2024-04-25T11:20:32.415957+0000","last_clean":"2024-04-25T11:20:32.415957+0000","last_became_active":"2024-04-25T11:20:31.453969+0000","last_became_peered":"2024-04-25T11:20:31.453969+0000","last_unstale":"2024-04-25T11:20:32.415957+0000","last_undegraded":"2024-04-25T11:20:32.415957+0000","last_fullsized":"2024-04-25T11:20:32.415957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:03.172347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.428251+0000","last_change":"2024-04-25T11:20:32.420333+0000","last_active":"2024-04-25T11:20:32.428251+0000","last_peered":"2024-04-25T11:20:32.428251+0000","last_clean":"2024-04-25T11:20:32.428251+0000","last_became_active":"2024-04-25T11:20:32.419345+0000","last_became_peered":"2024-04-25T11:20:32.419345+0000","last_unstale":"2024-04-25T11:20:32.428251+0000","last_undegraded":"2024-04-25T11:20:32.428251+0000","last_fullsized":"2024-04-25T11:20:32.428251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:51.320902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:33.901789+0000","last_change":"2024-04-25T11:20:31.446124+0000","last_active":"2024-04-25T11:20:33.901789+0000","last_peered":"2024-04-25T11:20:33.901789+0000","last_clean":"2024-04-25T11:20:33.901789+0000","last_became_active":"2024-04-25T11:20:31.445804+0000","last_became_peered":"2024-04-25T11:20:31.445804+0000","last_unstale":"2024-04-25T11:20:33.901789+0000","last_undegraded":"2024-04-25T11:20:33.901789+0000","last_fullsized":"2024-04-25T11:20:33.901789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:20.980572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.470297+0000","last_change":"2024-04-25T11:20:32.463197+0000","last_active":"2024-04-25T11:20:32.470297+0000","last_peered":"2024-04-25T11:20:32.470297+0000","last_clean":"2024-04-25T11:20:32.470297+0000","last_became_active":"2024-04-25T11:20:32.453859+0000","last_became_peered":"2024-04-25T11:20:32.453859+0000","last_unstale":"2024-04-25T11:20:32.470297+0000","last_undegraded":"2024-04-25T11:20:32.470297+0000","last_fullsized":"2024-04-25T11:20:32.470297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:31.392548+0000","last_clean_scrub_stamp":"2024-04-25T11:20:31.392548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:29.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.437075+0000","last_change":"2024-04-25T11:20:31.445115+0000","last_active":"2024-04-25T11:20:32.437075+0000","last_peered":"2024-04-25T11:20:32.437075+0000","last_clean":"2024-04-25T11:20:32.437075+0000","last_became_active":"2024-04-25T11:20:31.444279+0000","last_became_peered":"2024-04-25T11:20:31.444279+0000","last_unstale":"2024-04-25T11:20:32.437075+0000","last_undegraded":"2024-04-25T11:20:32.437075+0000","last_fullsized":"2024-04-25T11:20:32.437075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:30.382178+0000","last_clean_scrub_stamp":"2024-04-25T11:20:30.382178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:44.072745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":132,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:20:32.453166+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:20:32.453166+0000","last_peered":"2024-04-25T11:20:32.453166+0000","last_clean":"2024-04-25T11:20:32.453166+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:20:32.453166+0000","last_undegraded":"2024-04-25T11:20:32.453166+0000","last_fullsized":"2024-04-25T11:20:32.453166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:20:36.964 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:20:36.964 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:20:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16ecbcc640 1 Processor -- start 2024-04-25T11:20:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16ecbcc640 1 -- start start 2024-04-25T11:20:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16ecbcc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f16e80cda00 0x7f16e80b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16ecbcc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e80cdef0 con 0x7f16e80d0250 2024-04-25T11:20:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16ecbcc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e80ce1d0 con 0x7f16e80cd5f0 2024-04-25T11:20:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16ecbcc640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f16e80b5970 con 0x7f16e80cda00 2024-04-25T11:20:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16e659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f16e80d0250 0x7f16e80ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52582/0 (socket says 172.21.15.106:52582) 2024-04-25T11:20:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16e659c640 1 -- 172.21.15.106:0/1075373559 learned_addr learned my addr 172.21.15.106:0/1075373559 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16e6d9d640 1 --2- 172.21.15.106:0/1075373559 >> v2:172.21.15.134:3300/0 conn(0x7f16e80cda00 0x7f16e80b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2753043739 0 0) 0x7f16e80cdef0 con 0x7f16e80d0250 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.151+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16d0005740 con 0x7f16e80d0250 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 643363849 0 0) 0x7f16d0005740 con 0x7f16e80d0250 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 >> v2:172.21.15.134:3300/0 conn(0x7f16e80cda00 msgr2=0x7f16e80b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 --2- 172.21.15.106:0/1075373559 >> v2:172.21.15.134:3300/0 conn(0x7f16e80cda00 0x7f16e80b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 >> v1:172.21.15.134:6789/0 conn(0x7f16e80cd5f0 legacy=0x7f16e80b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e80b5320 con 0x7f16e80d0250 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f16e0001680 con 0x7f16e80d0250 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 >> v1:172.21.15.106:6789/0 conn(0x7f16e80d0250 legacy=0x7f16e80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16d0005bd0 0x7f16d0006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0006a20 0x7f16d0006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16d00073e0 0x7f16d000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f16e80b5970 con 0x7f16d0005bd0 2024-04-25T11:20:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16d000e380 con 0x7f16d0006a20 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f16d000e580 con 0x7f16d00073e0 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e659c640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16d0005bd0 0x7f16d0006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e6d9d640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16d00073e0 0x7f16d000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e659c640 1 -- 172.21.15.106:0/1075373559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16d00073e0 msgr2=0x7f16d000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e659c640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16d00073e0 0x7f16d000dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e659c640 1 -- 172.21.15.106:0/1075373559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0006a20 msgr2=0x7f16d0006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e659c640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0006a20 0x7f16d0006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e659c640 1 -- 172.21.15.106:0/1075373559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d0005740 con 0x7f16d0005bd0 2024-04-25T11:20:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e6d9d640 1 -- 172.21.15.106:0/1075373559 reap_dead start 2024-04-25T11:20:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e659c640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16d0005bd0 0x7f16d0006040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f16e000c210 tx=0x7f16e001cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8328a407b77151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0007d20 con 0x7f16d0005bd0 2024-04-25T11:20:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e00a9070 con 0x7f16d0005bd0 2024-04-25T11:20:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16e559a640 1 -- 172.21.15.106:0/1075373559 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0005da0 con 0x7f16d0005bd0 2024-04-25T11:20:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16ecbcc640 1 -- 172.21.15.106:0/1075373559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16d0005bd0 msgr2=0x7f16d0006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16ecbcc640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16d0005bd0 0x7f16d0006040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f16e000c210 tx=0x7f16e001cb20 comp rx=0 tx=0).stop 2024-04-25T11:20:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16ecbcc640 1 -- 172.21.15.106:0/1075373559 shutdown_connections 2024-04-25T11:20:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16ecbcc640 1 --2- 172.21.15.106:0/1075373559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16d0005bd0 0x7f16d0006040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16ecbcc640 1 -- 172.21.15.106:0/1075373559 >> 172.21.15.106:0/1075373559 conn(0x7f16e80b4020 msgr2=0x7f16e80b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16ecbcc640 1 -- 172.21.15.106:0/1075373559 shutdown_connections 2024-04-25T11:20:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.155+0000 7f16ecbcc640 1 -- 172.21.15.106:0/1075373559 wait complete. 2024-04-25T11:20:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 Processor -- start 2024-04-25T11:20:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 -- start start 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0005bd0 0x7f16e80c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16e80c9f30 0x7f16e80c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16e80c6ba0 0x7f16e80c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f16e80cd430 con 0x7f16e80c9f30 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16e80cd2b0 con 0x7f16d0005bd0 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f16e80cf280 con 0x7f16e80c6ba0 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16e80c9f30 0x7f16e80c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16e80c9f30 0x7f16e80c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60324/0 (socket says 172.21.15.106:60324) 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 -- 172.21.15.106:0/2135849962 learned_addr learned my addr 172.21.15.106:0/2135849962 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e659c640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0005bd0 0x7f16e80c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e6d9d640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16e80c6ba0 0x7f16e80c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 -- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16e80c6ba0 msgr2=0x7f16e80c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16e80c6ba0 0x7f16e80c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 -- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0005bd0 msgr2=0x7f16e80c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0005bd0 0x7f16e80c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 -- 172.21.15.106:0/2135849962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e80c74f0 con 0x7f16e80c9f30 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e659c640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0005bd0 0x7f16e80c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16e5d9b640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16e80c9f30 0x7f16e80c6660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c800 tx=0x7f16dc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16d6ffd640 1 -- 172.21.15.106:0/2135849962 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc003b80 con 0x7f16e80c9f30 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16d6ffd640 1 -- 172.21.15.106:0/2135849962 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc003d40 con 0x7f16e80c9f30 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e80c88c0 con 0x7f16e80c9f30 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16d6ffd640 1 -- 172.21.15.106:0/2135849962 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc00aa30 con 0x7f16e80c9f30 2024-04-25T11:20:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.159+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e80cacd0 con 0x7f16e80c9f30 2024-04-25T11:20:37.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.163+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e8050750 con 0x7f16e80c9f30 2024-04-25T11:20:37.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.163+0000 7f16d6ffd640 1 -- 172.21.15.106:0/2135849962 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f16dc00abd0 con 0x7f16e80c9f30 2024-04-25T11:20:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.163+0000 7f16d6ffd640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f16c806f400 0x7f16c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.163+0000 7f16d6ffd640 1 -- 172.21.15.106:0/2135849962 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f16dc08fcc0 con 0x7f16e80c9f30 2024-04-25T11:20:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.163+0000 7f16e659c640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f16c806f400 0x7f16c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.163+0000 7f16e659c640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f16c806f400 0x7f16c8071910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16e00ab000 tx=0x7f16e00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.167+0000 7f16d6ffd640 1 -- 172.21.15.106:0/2135849962 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f16dc05cc90 con 0x7f16e80c9f30 2024-04-25T11:20:37.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.367+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f16e80c5fa0 con 0x7f16e80c9f30 2024-04-25T11:20:37.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16d6ffd640 1 -- 172.21.15.106:0/2135849962 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f16dc05c590 con 0x7f16e80c9f30 2024-04-25T11:20:37.373 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-25T11:20:37.373 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADPCpmT+VXFxAASImB5WSW6aNZRALgQGZkFQ== 2024-04-25T11:20:37.373 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-25T11:20:37.373 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-25T11:20:37.373 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:20:37.374 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-25T11:20:37.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f16c806f400 msgr2=0x7f16c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f16c806f400 0x7f16c8071910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16e00ab000 tx=0x7f16e00a7070 comp rx=0 tx=0).stop 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16e80c9f30 msgr2=0x7f16e80c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16e80c9f30 0x7f16e80c6660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c800 tx=0x7f16dc00ccc0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 shutdown_connections 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16e80c6ba0 0x7f16e80c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f16c806f400 0x7f16c8071910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16e80c9f30 0x7f16e80c6660 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 --2- 172.21.15.106:0/2135849962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0005bd0 0x7f16e80c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 >> 172.21.15.106:0/2135849962 conn(0x7f16e80b4020 msgr2=0x7f16e815b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 shutdown_connections 2024-04-25T11:20:37.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.375+0000 7f16ecbcc640 1 -- 172.21.15.106:0/2135849962 wait complete. 2024-04-25T11:20:37.394 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:37.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2f605640 1 Processor -- start 2024-04-25T11:20:37.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2f605640 1 -- start start 2024-04-25T11:20:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2f605640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb280d2160 0x7fdb280d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2f605640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdb280b5120 0x7fdb280b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2f605640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb280d2ab0 con 0x7fdb280ba970 2024-04-25T11:20:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2f605640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdb280d1100 con 0x7fdb280b5120 2024-04-25T11:20:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2f605640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb280d0530 con 0x7fdb280d2160 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2d3a1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdb280ba970 0x7fdb280d1590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52584/0 (socket says 172.21.15.106:52584) 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2d3a1640 1 -- 172.21.15.106:0/2071154274 learned_addr learned my addr 172.21.15.106:0/2071154274 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2cba0640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb280d2160 0x7fdb280d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb2dba2640 1 --2- 172.21.15.106:0/2071154274 >> v2:172.21.15.134:3300/0 conn(0x7fdb280b5120 0x7fdb280b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581927777 0 0) 0x7fdb240012a0 con 0x7fdb280ba970 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.543+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb14003530 con 0x7fdb280ba970 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2496958229 0 0) 0x7fdb14003530 con 0x7fdb280ba970 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb280d2160 msgr2=0x7fdb280d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb280d2160 0x7fdb280d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 >> v2:172.21.15.134:3300/0 conn(0x7fdb280b5120 msgr2=0x7fdb280b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 --2- 172.21.15.106:0/2071154274 >> v2:172.21.15.134:3300/0 conn(0x7fdb280b5120 0x7fdb280b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb280d07d0 con 0x7fdb280ba970 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fdb240019d0 con 0x7fdb280ba970 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 >> v1:172.21.15.106:6789/0 conn(0x7fdb280ba970 legacy=0x7fdb280d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb14003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb14004810 0x7fdb14004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb280d2160 0x7fdb14005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb280d1100 con 0x7fdb140039c0 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb280d0530 con 0x7fdb14004810 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1400ba80 con 0x7fdb280d2160 2024-04-25T11:20:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2d3a1640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb14003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2dba2640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb280d2160 0x7fdb14005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2cba0640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb14004810 0x7fdb14004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2d3a1640 1 -- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb280d2160 msgr2=0x7fdb14005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:37.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2d3a1640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb280d2160 0x7fdb14005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2d3a1640 1 -- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb14004810 msgr2=0x7fdb14004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:37.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2d3a1640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb14004810 0x7fdb14004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2d3a1640 1 -- 172.21.15.106:0/2071154274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb14003530 con 0x7fdb140039c0 2024-04-25T11:20:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2d3a1640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb14003e30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdb280b8320 tx=0x7fdb2401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b01425af3521ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400c0e0 con 0x7fdb140039c0 2024-04-25T11:20:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb240a9020 con 0x7fdb140039c0 2024-04-25T11:20:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb1ffff640 1 -- 172.21.15.106:0/2071154274 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24007310 con 0x7fdb140039c0 2024-04-25T11:20:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2f605640 1 -- 172.21.15.106:0/2071154274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 msgr2=0x7fdb14003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2f605640 1 --2- 172.21.15.106:0/2071154274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb14003e30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdb280b8320 tx=0x7fdb2401cc40 comp rx=0 tx=0).stop 2024-04-25T11:20:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2dba2640 1 -- 172.21.15.106:0/2071154274 reap_dead start 2024-04-25T11:20:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2f605640 1 -- 172.21.15.106:0/2071154274 shutdown_connections 2024-04-25T11:20:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2f605640 1 -- 172.21.15.106:0/2071154274 >> 172.21.15.106:0/2071154274 conn(0x7fdb280d57e0 msgr2=0x7fdb280d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2f605640 1 -- 172.21.15.106:0/2071154274 shutdown_connections 2024-04-25T11:20:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.547+0000 7fdb2f605640 1 -- 172.21.15.106:0/2071154274 wait complete. 2024-04-25T11:20:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 Processor -- start 2024-04-25T11:20:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 -- start start 2024-04-25T11:20:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb2815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb281fdba0 0x7fdb2815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb281fcfe0 0x7fdb281fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb280bb200 con 0x7fdb140039c0 2024-04-25T11:20:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb280bb080 con 0x7fdb281fcfe0 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb280b5a70 con 0x7fdb281fdba0 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb2815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb2815cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60336/0 (socket says 172.21.15.106:60336) 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 -- 172.21.15.106:0/2646028164 learned_addr learned my addr 172.21.15.106:0/2646028164 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2dba2640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb281fcfe0 0x7fdb281fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2cba0640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb281fdba0 0x7fdb2815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 -- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb281fdba0 msgr2=0x7fdb2815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb281fdba0 0x7fdb2815ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 -- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb281fcfe0 msgr2=0x7fdb281fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb281fcfe0 0x7fdb281fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 -- 172.21.15.106:0/2646028164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb281fd930 con 0x7fdb140039c0 2024-04-25T11:20:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2dba2640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb281fcfe0 0x7fdb281fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2d3a1640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb2815cf70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdb24000f00 tx=0x7fdb14005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb1dffb640 1 -- 172.21.15.106:0/2646028164 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240070d0 con 0x7fdb140039c0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb1dffb640 1 -- 172.21.15.106:0/2646028164 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb24007270 con 0x7fdb140039c0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.551+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2815f2d0 con 0x7fdb140039c0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.555+0000 7fdb1dffb640 1 -- 172.21.15.106:0/2646028164 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240057b0 con 0x7fdb140039c0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.555+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb2815f890 con 0x7fdb140039c0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.555+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb280c8c60 con 0x7fdb140039c0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.555+0000 7fdb1dffb640 1 -- 172.21.15.106:0/2646028164 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb24005950 con 0x7fdb140039c0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.555+0000 7fdb1dffb640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb0c06f400 0x7fdb0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.555+0000 7fdb1dffb640 1 -- 172.21.15.106:0/2646028164 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdb24128360 con 0x7fdb140039c0 2024-04-25T11:20:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.555+0000 7fdb2cba0640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb0c06f400 0x7fdb0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.555+0000 7fdb2cba0640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb0c06f400 0x7fdb0c071910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb28166f10 tx=0x7fdb18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.559+0000 7fdb1dffb640 1 -- 172.21.15.106:0/2646028164 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb281fd930 con 0x7fdb140039c0 2024-04-25T11:20:37.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.755+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb280ba410 con 0x7fdb140039c0 2024-04-25T11:20:37.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.755+0000 7fdb1dffb640 1 -- 172.21.15.106:0/2646028164 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fdb240b4070 con 0x7fdb140039c0 2024-04-25T11:20:37.755 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:37.755 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3211345151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3211345151},{"type":"v1","addr":"172.21.15.134:6833","nonce":3211345151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1962020849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1962020849},{"type":"v1","addr":"172.21.15.106:6835","nonce":1962020849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3211705687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3211705687},{"type":"v1","addr":"172.21.15.134:6835","nonce":3211705687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:31.811623+0000","modified":"2024-04-25T11:20:34.872100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1141169456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1141169456},{"type":"v1","addr":"172.21.15.106:6837","nonce":1141169456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T11:20:37.756 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb0c06f400 msgr2=0x7fdb0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb0c06f400 0x7fdb0c071910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb28166f10 tx=0x7fdb18008040 comp rx=0 tx=0).stop 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 msgr2=0x7fdb2815cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb2815cf70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdb24000f00 tx=0x7fdb14005840 comp rx=0 tx=0).stop 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 shutdown_connections 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb281fcfe0 0x7fdb281fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb281fdba0 0x7fdb2815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb0c06f400 0x7fdb0c071910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 --2- 172.21.15.106:0/2646028164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb140039c0 0x7fdb2815cf70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 >> 172.21.15.106:0/2646028164 conn(0x7fdb280d57e0 msgr2=0x7fdb2816a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 shutdown_connections 2024-04-25T11:20:37.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:37.759+0000 7fdb2f605640 1 -- 172.21.15.106:0/2646028164 wait complete. 2024-04-25T11:20:38.179 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:20:31.811623+0000', 'modified': '2024-04-25T11:20:34.872100+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5214}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5214': {'gid': 5214, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6837/1141169456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 1141169456}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 1141169456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5214, 'qdb_cluster': [5214]} 2024-04-25T11:20:38.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:20:38.180 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:20:38.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f801b783640 1 Processor -- start 2024-04-25T11:20:38.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f801b783640 1 -- start start 2024-04-25T11:20:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f801b783640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f801406dd10 0x7f801415c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f801b783640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801415d090 con 0x7f8014153230 2024-04-25T11:20:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f801b783640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80140d17f0 con 0x7f801415cc80 2024-04-25T11:20:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f801b783640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8014153640 con 0x7f801406dd10 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f801951f640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8014153230 0x7f801406d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52594/0 (socket says 172.21.15.106:52594) 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f801951f640 1 -- 172.21.15.106:0/3842932216 learned_addr learned my addr 172.21.15.106:0/3842932216 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8018d1e640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f801406dd10 0x7f801415c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 372686472 0 0) 0x7f801415d090 con 0x7f8014153230 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ffc004640 con 0x7f8014153230 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1723004259 0 0) 0x7f7ffc004640 con 0x7f8014153230 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f801406dd10 msgr2=0x7f801415c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8013fff640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f801406dd10 0x7f801415c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 >> v1:172.21.15.134:6789/0 conn(0x7f801415cc80 legacy=0x7f8014070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801415d600 con 0x7f8014153230 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.339+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f800c001a00 con 0x7f8014153230 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 >> v1:172.21.15.106:6789/0 conn(0x7f8014153230 legacy=0x7f801406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ffc004b00 0x7f7ffc004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc005490 0x7f7ffc0059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ffc0061e0 0x7f7ffc00cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc00d050 con 0x7f7ffc0061e0 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc00d200 con 0x7f7ffc005490 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc0035b0 con 0x7f7ffc004b00 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8019d20640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ffc0061e0 0x7f7ffc00cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f801951f640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ffc004b00 0x7f7ffc004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8018d1e640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc005490 0x7f7ffc0059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8019d20640 1 -- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ffc004b00 msgr2=0x7f7ffc004f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8019d20640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ffc004b00 0x7f7ffc004f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8019d20640 1 -- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc005490 msgr2=0x7f7ffc0059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8019d20640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc005490 0x7f7ffc0059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8019d20640 1 -- 172.21.15.106:0/3842932216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc004640 con 0x7f7ffc0061e0 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8018d1e640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc005490 0x7f7ffc0059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:38.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f801951f640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ffc004b00 0x7f7ffc004f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8019d20640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ffc0061e0 0x7f7ffc00cae0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f801406f080 tx=0x7f800401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3cb26c4cdc8d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80040013a0 con 0x7f7ffc0061e0 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80040052f0 con 0x7f7ffc0061e0 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.343+0000 7f8013fff640 1 -- 172.21.15.106:0/3842932216 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004003840 con 0x7f7ffc0061e0 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- 172.21.15.106:0/3842932216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ffc0061e0 msgr2=0x7f7ffc00cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 --2- 172.21.15.106:0/3842932216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ffc0061e0 0x7f7ffc00cae0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f801406f080 tx=0x7f800401c380 comp rx=0 tx=0).stop 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f8019d20640 1 -- 172.21.15.106:0/3842932216 reap_dead start 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- 172.21.15.106:0/3842932216 shutdown_connections 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- 172.21.15.106:0/3842932216 >> 172.21.15.106:0/3842932216 conn(0x7f8014073300 msgr2=0x7f8014073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- 172.21.15.106:0/3842932216 shutdown_connections 2024-04-25T11:20:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- 172.21.15.106:0/3842932216 wait complete. 2024-04-25T11:20:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 Processor -- start 2024-04-25T11:20:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- start start 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f801409bf90 0x7f80140b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80140b49f0 0x7f80141457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80140b4e00 0x7f801406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80140cfb40 con 0x7f80140b49f0 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80140cf860 con 0x7f801409bf90 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f801b783640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f80140cfe20 con 0x7f80140b4e00 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f8018d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80140b49f0 0x7f80141457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f8018d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80140b49f0 0x7f80141457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60354/0 (socket says 172.21.15.106:60354) 2024-04-25T11:20:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.347+0000 7f8018d1e640 1 -- 172.21.15.106:0/320998604 learned_addr learned my addr 172.21.15.106:0/320998604 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f801951f640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f801409bf90 0x7f80140b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8019d20640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80140b4e00 0x7f801406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8018d1e640 1 -- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80140b4e00 msgr2=0x7f801406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8018d1e640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80140b4e00 0x7f801406ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8018d1e640 1 -- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f801409bf90 msgr2=0x7f80140b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8018d1e640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f801409bf90 0x7f80140b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8018d1e640 1 -- 172.21.15.106:0/320998604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80140b52f0 con 0x7f80140b49f0 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f801951f640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f801409bf90 0x7f80140b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8019d20640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80140b4e00 0x7f801406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8018d1e640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80140b49f0 0x7f80141457c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8008009fd0 tx=0x7f8008011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8011ffb640 1 -- 172.21.15.106:0/320998604 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008004220 con 0x7f80140b49f0 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8011ffb640 1 -- 172.21.15.106:0/320998604 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80080043c0 con 0x7f80140b49f0 2024-04-25T11:20:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f801b783640 1 -- 172.21.15.106:0/320998604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80140b55d0 con 0x7f80140b49f0 2024-04-25T11:20:38.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8011ffb640 1 -- 172.21.15.106:0/320998604 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800f9e0 con 0x7f80140b49f0 2024-04-25T11:20:38.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f801b783640 1 -- 172.21.15.106:0/320998604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801409e7c0 con 0x7f80140b49f0 2024-04-25T11:20:38.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8011ffb640 1 -- 172.21.15.106:0/320998604 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f800800fb80 con 0x7f80140b49f0 2024-04-25T11:20:38.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8011ffb640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f800006f430 0x7f8000071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f801951f640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f800006f430 0x7f8000071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f8011ffb640 1 -- 172.21.15.106:0/320998604 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f800808f8e0 con 0x7f80140b49f0 2024-04-25T11:20:38.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.351+0000 7f801951f640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f800006f430 0x7f8000071940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f801406f2b0 tx=0x7f800c001280 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 Processor -- start 2024-04-25T11:20:38.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- start start 2024-04-25T11:20:38.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c4a15ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c48a3ea0 0x5635c4a16420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c449c280 0x5635c4a16960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5635c4885160 con 0x5635c48b7c20 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5635c481ae20 con 0x5635c449c280 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5635c47d3a20 con 0x5635c48a3ea0 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c4a15ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c4a15ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60360/0 (socket says 172.21.15.106:60360) 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 -- 172.21.15.106:0/899443296 learned_addr learned my addr 172.21.15.106:0/899443296 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8019d20640 1 --2- 172.21.15.106:0/899443296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c449c280 0x5635c4a16960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 -- 172.21.15.106:0/899443296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c48a3ea0 msgr2=0x5635c4a16420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 --2- 172.21.15.106:0/899443296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c48a3ea0 0x5635c4a16420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 -- 172.21.15.106:0/899443296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c449c280 msgr2=0x5635c4a16960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 --2- 172.21.15.106:0/899443296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c449c280 0x5635c4a16960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 -- 172.21.15.106:0/899443296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c47d8100 con 0x5635c48b7c20 2024-04-25T11:20:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f801951f640 1 --2- 172.21.15.106:0/899443296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c4a15ee0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f800c0021a0 tx=0x7f800c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778f4073dd65c06a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f7ffaffd640 1 -- 172.21.15.106:0/899443296 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c025070 con 0x5635c48b7c20 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f7ffaffd640 1 -- 172.21.15.106:0/899443296 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c01b050 con 0x5635c48b7c20 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f7ffaffd640 1 -- 172.21.15.106:0/899443296 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c010040 con 0x5635c48b7c20 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- 172.21.15.106:0/899443296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 msgr2=0x5635c4a15ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 --2- 172.21.15.106:0/899443296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c4a15ee0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f800c0021a0 tx=0x7f800c012000 comp rx=0 tx=0).stop 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- 172.21.15.106:0/899443296 shutdown_connections 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 --2- 172.21.15.106:0/899443296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c449c280 0x5635c4a16960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 --2- 172.21.15.106:0/899443296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c48a3ea0 0x5635c4a16420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 --2- 172.21.15.106:0/899443296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c4a15ee0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- 172.21.15.106:0/899443296 >> 172.21.15.106:0/899443296 conn(0x5635c44dbff0 msgr2=0x5635c49cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- 172.21.15.106:0/899443296 shutdown_connections 2024-04-25T11:20:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.355+0000 7f8022ae0000 1 -- 172.21.15.106:0/899443296 wait complete. 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 -- start start 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 Processor -- start 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 0x5635c4a1ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 0x5635c4a1d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c49d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5635c4828220 con 0x5635c48b7c20 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5635c47beea0 con 0x5635c48a3ea0 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5635c4815b00 con 0x5635c449c280 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 10 client.0 _ll_register_callbacks cb 0x5635c4827dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:20:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c49d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c49d80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60376/0 (socket says 172.21.15.106:60376) 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 -- 172.21.15.106:0/3913171701 learned_addr learned my addr 172.21.15.106:0/3913171701 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f801951f640 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 0x5635c4a1ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8018d1e640 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 0x5635c4a1d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 -- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 msgr2=0x5635c4a1ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 0x5635c4a1ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 -- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 msgr2=0x5635c4a1d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 0x5635c4a1d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 -- 172.21.15.106:0/3913171701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c47ba500 con 0x5635c48b7c20 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8018d1e640 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 0x5635c4a1d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8019d20640 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c49d80c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f800401c1b0 tx=0x7f8004019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b23d6758f21167 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f7ff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T11:20:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f7ff3fff640 1 -- 172.21.15.106:0/3913171701 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80040ba060 con 0x5635c48b7c20 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f7ff3fff640 1 -- 172.21.15.106:0/3913171701 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8004007230 con 0x5635c48b7c20 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 -- 172.21.15.106:0/3913171701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800c01b050 con 0x5635c48b7c20 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f7ff3fff640 1 -- 172.21.15.106:0/3913171701 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f80040c0050 con 0x5635c48b7c20 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 10 client.5382 fetch_fsmap learned FSMap version 17 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 -- 172.21.15.106:0/3913171701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5635c47a0ea0 con 0x5635c48b7c20 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f7ff3fff640 1 -- 172.21.15.106:0/3913171701 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c01b050 con 0x5635c48b7c20 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f7ff3fff640 1 -- 172.21.15.106:0/3913171701 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f8004007430 con 0x5635c48b7c20 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 10 client.5382 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 10 client.5382 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 20 client.5382 populate_metadata read hostname 'smithi106' 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] conn(0x5635c44c07d0 0x5635c49d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 4 client.5382 mds_command: new command op to 5214 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f8022ae0000 1 -- 172.21.15.106:0/3913171701 --> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635c47b3be0 con 0x5635c44c07d0 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f801951f640 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] conn(0x5635c44c07d0 0x5635c49d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f7ff3fff640 10 client.5382 ms_handle_connect on v2:172.21.15.106:6836/1141169456 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.359+0000 7f801951f640 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] conn(0x5635c44c07d0 0x5635c49d8e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c9c308152240ff server_cookie=c58cab559659a257 in_seq=0 out_seq=0 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f7ff3fff640 1 -- 172.21.15.106:0/3913171701 <== mds.0 v2:172.21.15.106:6836/1141169456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5635c47b3be0 con 0x5635c44c07d0 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f7ff3fff640 10 client.5382 handle_command_reply: tid=0 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 client.5382 shutdown 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 -- 172.21.15.106:0/3913171701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 msgr2=0x5635c49d80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c49d80c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f800401c1b0 tx=0x7f8004019120 comp rx=0 tx=0).stop 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 -- 172.21.15.106:0/3913171701 shutdown_connections 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] conn(0x5635c44c07d0 0x5635c49d8e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48b7c20 0x5635c49d80c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f7ff3fff640 0 client.5382 ms_handle_reset on v2:172.21.15.106:6836/1141169456 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 0x5635c4a1d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 --2- 172.21.15.106:0/3913171701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 0x5635c4a1ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 -- 172.21.15.106:0/3913171701 >> 172.21.15.106:0/3913171701 conn(0x5635c44dbff0 msgr2=0x5635c49cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 -- 172.21.15.106:0/3913171701 shutdown_connections 2024-04-25T11:20:38.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 1 -- 172.21.15.106:0/3913171701 wait complete. 2024-04-25T11:20:38.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.363+0000 7f8022ae0000 20 client.5382 trim_cache size 0 max 16384 2024-04-25T11:20:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f8022ae0000 1 Processor -- start 2024-04-25T11:20:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f8022ae0000 1 -- start start 2024-04-25T11:20:38.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c449c280 0x5635c4a220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 0x5635c4a2bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c48b7c20 0x5635c4a2c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f8022ae0000 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80141fde60 con 0x5635c449c280 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f8022ae0000 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80141f5eb0 con 0x5635c48a3ea0 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f8022ae0000 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f80141fd8c0 con 0x5635c48b7c20 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f801951f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c449c280 0x5635c4a220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f801951f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c449c280 0x5635c4a220b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60388/0 (socket says 172.21.15.106:60388) 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f801951f640 1 -- 172.21.15.106:0/869643477 learned_addr learned my addr 172.21.15.106:0/869643477 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f801951f640 1 -- 172.21.15.106:0/869643477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c48b7c20 msgr2=0x5635c4a2c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f801951f640 1 --2- 172.21.15.106:0/869643477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c48b7c20 0x5635c4a2c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f801951f640 1 -- 172.21.15.106:0/869643477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 msgr2=0x5635c4a2bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f801951f640 1 --2- 172.21.15.106:0/869643477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 0x5635c4a2bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.379+0000 7f801951f640 1 -- 172.21.15.106:0/869643477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c49d8430 con 0x5635c449c280 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f801951f640 1 --2- 172.21.15.106:0/869643477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c449c280 0x5635c4a220b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f800c000f90 tx=0x7f800c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dd942f4ce1daae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f7ffa7fc640 1 -- 172.21.15.106:0/869643477 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c010040 con 0x5635c449c280 2024-04-25T11:20:38.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f7ffa7fc640 1 -- 172.21.15.106:0/869643477 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c01b040 con 0x5635c449c280 2024-04-25T11:20:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f7ffa7fc640 1 -- 172.21.15.106:0/869643477 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c025040 con 0x5635c449c280 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- 172.21.15.106:0/869643477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c449c280 msgr2=0x5635c4a220b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 --2- 172.21.15.106:0/869643477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c449c280 0x5635c4a220b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f800c000f90 tx=0x7f800c014000 comp rx=0 tx=0).stop 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- 172.21.15.106:0/869643477 shutdown_connections 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 --2- 172.21.15.106:0/869643477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c48b7c20 0x5635c4a2c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 --2- 172.21.15.106:0/869643477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48a3ea0 0x5635c4a2bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 --2- 172.21.15.106:0/869643477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c449c280 0x5635c4a220b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- 172.21.15.106:0/869643477 >> 172.21.15.106:0/869643477 conn(0x5635c44dbff0 msgr2=0x5635c49d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- 172.21.15.106:0/869643477 shutdown_connections 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- 172.21.15.106:0/869643477 wait complete. 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- start start 2024-04-25T11:20:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 Processor -- start 2024-04-25T11:20:38.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 0x5635c4a887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48a3ea0 0x5635c4a88ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48b7c20 0x5635c4a209b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5635c481a130 con 0x5635c48a3ea0 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5635c47b07d0 con 0x5635c48b7c20 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5635c481f450 con 0x5635c449c280 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8022ae0000 10 client.0 _ll_register_callbacks cb 0x7f80040a4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48a3ea0 0x5635c4a88ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48a3ea0 0x5635c4a88ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60398/0 (socket says 172.21.15.106:60398) 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 -- 172.21.15.106:0/1682112706 learned_addr learned my addr 172.21.15.106:0/1682112706 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 -- 172.21.15.106:0/1682112706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 msgr2=0x5635c4a887a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 0x5635c4a887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 -- 172.21.15.106:0/1682112706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48b7c20 msgr2=0x5635c4a209b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48b7c20 0x5635c4a209b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 -- 172.21.15.106:0/1682112706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c47d3c70 con 0x5635c48a3ea0 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f8018d1e640 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48a3ea0 0x5635c4a88ce0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8008009160 tx=0x7f800805c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bf6fd2df90baf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.383+0000 7f7ff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T11:20:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 1 -- 172.21.15.106:0/1682112706 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008075020 con 0x5635c48a3ea0 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 1 -- 172.21.15.106:0/1682112706 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8008057d60 con 0x5635c48a3ea0 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 -- 172.21.15.106:0/1682112706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800c01b040 con 0x5635c48a3ea0 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 1 -- 172.21.15.106:0/1682112706 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8008022040 con 0x5635c48a3ea0 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 10 client.5388 fetch_fsmap learned FSMap version 17 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 -- 172.21.15.106:0/1682112706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5635c489d180 con 0x5635c48a3ea0 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 1 -- 172.21.15.106:0/1682112706 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c01b040 con 0x5635c48a3ea0 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 1 -- 172.21.15.106:0/1682112706 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f800805aec0 con 0x5635c48a3ea0 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 10 client.5388 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 10 client.5388 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 20 client.5388 populate_metadata read hostname 'smithi106' 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] conn(0x5635c44ae830 0x5635c4a91610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 4 client.5388 mds_command: new command op to 5214 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 -- 172.21.15.106:0/1682112706 --> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] -- command(tid 0: {"prefix": "status"}) -- 0x5635c4828a30 con 0x5635c44ae830 2024-04-25T11:20:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801951f640 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] conn(0x5635c44ae830 0x5635c4a91610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 10 client.5388 ms_handle_connect on v2:172.21.15.106:6836/1141169456 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801951f640 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] conn(0x5635c44ae830 0x5635c4a91610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bc705e33fa48d9 server_cookie=7a5488189e15e7f4 in_seq=0 out_seq=0 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 1 -- 172.21.15.106:0/1682112706 <== mds.0 v2:172.21.15.106:6836/1141169456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5635c4828a30 con 0x5635c44ae830 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 10 client.5388 handle_command_reply: tid=0 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 client.5388 shutdown 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 -- 172.21.15.106:0/1682112706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48a3ea0 msgr2=0x5635c4a88ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48a3ea0 0x5635c4a88ce0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8008009160 tx=0x7f800805c810 comp rx=0 tx=0).stop 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 -- 172.21.15.106:0/1682112706 shutdown_connections 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.106:6836/1141169456,v1:172.21.15.106:6837/1141169456] conn(0x5635c44ae830 0x5635c4a91610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635c48b7c20 0x5635c4a209b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5635c48a3ea0 0x5635c4a88ce0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 --2- 172.21.15.106:0/1682112706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5635c449c280 0x5635c4a887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f7ff3fff640 0 client.5388 ms_handle_reset on v2:172.21.15.106:6836/1141169456 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 -- 172.21.15.106:0/1682112706 >> 172.21.15.106:0/1682112706 conn(0x5635c44dbff0 msgr2=0x5635c4a1b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 -- 172.21.15.106:0/1682112706 shutdown_connections 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 1 -- 172.21.15.106:0/1682112706 wait complete. 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f8022ae0000 20 client.5388 trim_cache size 0 max 16384 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "8efdb3a5-72cb-436f-9be5-cf4e40adf704", 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5214, 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-25T11:20:38.392 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-25T11:20:38.393 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 5.9833644369999996, 2024-04-25T11:20:38.393 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 17, 2024-04-25T11:20:38.393 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 33, 2024-04-25T11:20:38.393 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 33, 2024-04-25T11:20:38.393 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 13.839151807 2024-04-25T11:20:38.393 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 -- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f800006f430 msgr2=0x7f8000071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f800006f430 0x7f8000071940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f801406f2b0 tx=0x7f800c001280 comp rx=0 tx=0).stop 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 -- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80140b49f0 msgr2=0x7f80141457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80140b49f0 0x7f80141457c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8008009fd0 tx=0x7f8008011b30 comp rx=0 tx=0).stop 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 -- 172.21.15.106:0/320998604 shutdown_connections 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80140b4e00 0x7f801406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f800006f430 0x7f8000071940 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80140b49f0 0x7f80141457c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 --2- 172.21.15.106:0/320998604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f801409bf90 0x7f80140b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 -- 172.21.15.106:0/320998604 >> 172.21.15.106:0/320998604 conn(0x7f8014073300 msgr2=0x7f801406ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 -- 172.21.15.106:0/320998604 shutdown_connections 2024-04-25T11:20:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.387+0000 7f801b783640 1 -- 172.21.15.106:0/320998604 wait complete. 2024-04-25T11:20:38.405 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:20:38.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c3ec4640 1 Processor -- start 2024-04-25T11:20:38.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c3ec4640 1 -- start start 2024-04-25T11:20:38.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c3ec4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f37bc0cef60 0x7f37bc073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c3ec4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37bc0ce050 con 0x7f37bc15e0a0 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c3ec4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37bc15e6b0 con 0x7f37bc0d0ce0 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c3ec4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f37bc15e990 con 0x7f37bc0cef60 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c1c60640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f37bc15e0a0 0x7f37bc0ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52602/0 (socket says 172.21.15.106:52602) 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c1c60640 1 -- 172.21.15.106:0/421325837 learned_addr learned my addr 172.21.15.106:0/421325837 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c145f640 1 --2- 172.21.15.106:0/421325837 >> v2:172.21.15.134:3300/0 conn(0x7f37bc0cef60 0x7f37bc073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 904471306 0 0) 0x7f37bc0ce050 con 0x7f37bc15e0a0 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.547+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37a8003530 con 0x7f37bc15e0a0 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 302678380 0 0) 0x7f37a8003530 con 0x7f37bc15e0a0 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 >> v2:172.21.15.134:3300/0 conn(0x7f37bc0cef60 msgr2=0x7f37bc073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 --2- 172.21.15.106:0/421325837 >> v2:172.21.15.134:3300/0 conn(0x7f37bc0cef60 0x7f37bc073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 >> v1:172.21.15.134:6789/0 conn(0x7f37bc0d0ce0 legacy=0x7f37bc0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc15ec70 con 0x7f37bc15e0a0 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f37b8001680 con 0x7f37bc15e0a0 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 >> v1:172.21.15.106:6789/0 conn(0x7f37bc15e0a0 legacy=0x7f37bc0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a80039c0 0x7f37a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004810 0x7f37a8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a80051d0 0x7f37a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc15e990 con 0x7f37a80051d0 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37a800c170 con 0x7f37a80039c0 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f37a800c370 con 0x7f37a8004810 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c2461640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a80051d0 0x7f37a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c1c60640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a80039c0 0x7f37a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c145f640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004810 0x7f37a8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c2461640 1 -- 172.21.15.106:0/421325837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004810 msgr2=0x7f37a8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c2461640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004810 0x7f37a8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c2461640 1 -- 172.21.15.106:0/421325837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a80039c0 msgr2=0x7f37a8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:38.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c2461640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a80039c0 0x7f37a8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c2461640 1 -- 172.21.15.106:0/421325837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a8003530 con 0x7f37a80051d0 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c2461640 1 -- 172.21.15.106:0/421325837 reap_dead start 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c2461640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a80051d0 0x7f37a800b9d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f37b4001e70 tx=0x7f37b401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385b9d76ac102682 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4001a90 con 0x7f37a80051d0 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b4003bc0 con 0x7f37a80051d0 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c0c5e640 1 -- 172.21.15.106:0/421325837 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40050d0 con 0x7f37a80051d0 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c3ec4640 1 -- 172.21.15.106:0/421325837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a80051d0 msgr2=0x7f37a800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c3ec4640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a80051d0 0x7f37a800b9d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f37b4001e70 tx=0x7f37b401c6e0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c3ec4640 1 -- 172.21.15.106:0/421325837 shutdown_connections 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c3ec4640 1 --2- 172.21.15.106:0/421325837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a80051d0 0x7f37a800b9d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c3ec4640 1 -- 172.21.15.106:0/421325837 >> 172.21.15.106:0/421325837 conn(0x7f37bc06c7c0 msgr2=0x7f37bc06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c3ec4640 1 -- 172.21.15.106:0/421325837 shutdown_connections 2024-04-25T11:20:38.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.551+0000 7f37c3ec4640 1 -- 172.21.15.106:0/421325837 wait complete. 2024-04-25T11:20:38.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 Processor -- start 2024-04-25T11:20:38.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 -- start start 2024-04-25T11:20:38.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a80051d0 0x7f37bc1f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc13f8b0 0x7f37bc0b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0b17b0 0x7f37bc13b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc0d06e0 con 0x7f37bc13f8b0 2024-04-25T11:20:38.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc15e4b0 con 0x7f37bc0b17b0 2024-04-25T11:20:38.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f37bc0d0860 con 0x7f37a80051d0 2024-04-25T11:20:38.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc13f8b0 0x7f37bc0b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc13f8b0 0x7f37bc0b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60426/0 (socket says 172.21.15.106:60426) 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 -- 172.21.15.106:0/3181276313 learned_addr learned my addr 172.21.15.106:0/3181276313 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c1c60640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a80051d0 0x7f37bc1f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c2461640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0b17b0 0x7f37bc13b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 -- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a80051d0 msgr2=0x7f37bc1f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a80051d0 0x7f37bc1f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 -- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0b17b0 msgr2=0x7f37bc13b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0b17b0 0x7f37bc13b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 -- 172.21.15.106:0/3181276313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc139840 con 0x7f37bc13f8b0 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c2461640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0b17b0 0x7f37bc13b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c1c60640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a80051d0 0x7f37bc1f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c145f640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc13f8b0 0x7f37bc0b1270 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f37ac011590 tx=0x7f37ac011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37b27fc640 1 -- 172.21.15.106:0/3181276313 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac015870 con 0x7f37bc13f8b0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37b27fc640 1 -- 172.21.15.106:0/3181276313 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ac015a10 con 0x7f37bc13f8b0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc1f6c60 con 0x7f37bc13f8b0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37b27fc640 1 -- 172.21.15.106:0/3181276313 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac015dc0 con 0x7f37bc13f8b0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.555+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc139ab0 con 0x7f37bc13f8b0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.559+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37bc142910 con 0x7f37bc13f8b0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.559+0000 7f37b27fc640 1 -- 172.21.15.106:0/3181276313 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f37ac013400 con 0x7f37bc13f8b0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.559+0000 7f37b27fc640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f37a006f430 0x7f37a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.559+0000 7f37c1c60640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f37a006f430 0x7f37a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.559+0000 7f37b27fc640 1 -- 172.21.15.106:0/3181276313 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f37ac08faf0 con 0x7f37bc13f8b0 2024-04-25T11:20:38.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.559+0000 7f37c1c60640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f37a006f430 0x7f37a0071940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37b80058c0 tx=0x7f37b8000ee0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.563+0000 7f37b27fc640 1 -- 172.21.15.106:0/3181276313 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f37bc1f6c60 con 0x7f37bc13f8b0 2024-04-25T11:20:38.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f37bc06cf50 con 0x7f37bc13f8b0 2024-04-25T11:20:38.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37b27fc640 1 -- 172.21.15.106:0/3181276313 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f37ac019030 con 0x7f37bc13f8b0 2024-04-25T11:20:38.746 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:20:38.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f37a006f430 msgr2=0x7f37a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f37a006f430 0x7f37a0071940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37b80058c0 tx=0x7f37b8000ee0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc13f8b0 msgr2=0x7f37bc0b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc13f8b0 0x7f37bc0b1270 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f37ac011590 tx=0x7f37ac011a50 comp rx=0 tx=0).stop 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 shutdown_connections 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0b17b0 0x7f37bc13b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f37a006f430 0x7f37a0071940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc13f8b0 0x7f37bc0b1270 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 --2- 172.21.15.106:0/3181276313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a80051d0 0x7f37bc1f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 >> 172.21.15.106:0/3181276313 conn(0x7f37bc06c7c0 msgr2=0x7f37bc06fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 shutdown_connections 2024-04-25T11:20:38.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38.747+0000 7f37c3ec4640 1 -- 172.21.15.106:0/3181276313 wait complete. 2024-04-25T11:20:38.766 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:20:38.766 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:20:38.766 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-25T11:20:38.766 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:20:38.766 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:38.766 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:20:38.766 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:20:38.766 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:20:38.767 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:38.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:38.767 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-25T11:20:38.773 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:20:38.773 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:20:38.773 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:20:38.773 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:38.773 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-25T11:20:38.773 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:38.773 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:20:38.773 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42445sec preferred_lft 42445sec 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T11:20:38.774 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 02:42:4f:a1:63:df brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:38.775 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T11:20:38.776 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:38.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:38.776 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:20:38.825 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-25T11:20:38.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:38.827 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-25T11:20:38.877 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:20:38.878 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:20:38.879 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:20:38.879 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:20:38.879 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:20:38.879 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:20:38.879 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:20:38.879 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:20:38.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:20:38.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:38.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:38.998 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T11:20:39.045 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:20:39.045 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:20:39.045 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:39.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:39.046 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-25T11:20:39.104 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T11:20:39.104 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:20:39.104 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:39.150 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:39.150 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:20:39.150 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:39.197 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:20:39.255 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:20:39.303 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:20:39.304 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:20:39.305 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:20:39.306 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:20:39.306 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:20:39.306 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:20:39.306 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:39.306 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:39.306 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:39.306 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:39.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:39.306 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:20:39.349 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T11:20:39.350 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T11:20:39.395 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:20:39.409 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:20:39.458 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:20:39.458 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:20:39.458 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:20:39.458 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:20:39.458 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:20:39.459 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:39.460 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:39.461 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:39.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:39.461 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:20:39.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:2024-04-25T11:20:39.503+0000 7ffa3bd29580 -1 init, newargv = 0x7ffa280058b0 newargc=15 2024-04-25T11:20:39.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[38150]: starting ceph client 2024-04-25T11:20:39.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[38150]: starting fuse 2024-04-25T11:20:40.607 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:20:40.623 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:20:40.671 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:20:40.672 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:40.673 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:40.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:40.674 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:20:40.721 INFO:teuthology.orchestra.run.smithi106.stdout:45 2024-04-25T11:20:40.721 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T11:20:40.721 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:20:40.721 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-25T11:20:40.722 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-25T11:20:40.723 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-25T11:20:40.846 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.38150.asok 2024-04-25T11:20:40.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:40.858 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38150.asok 2024-04-25T11:20:40.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:40.858 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38150.asok status 2024-04-25T11:20:41.034 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "0", 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "38150", 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-25T11:20:41.035 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5400, 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 5400 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 627962865 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T11:20:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 627962865 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.5400 192.168.0.1:0/627962865", 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/627962865", 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 17, 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 33, 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-25T11:20:41.037 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:20:41.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:41.050 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-25T11:20:41.050 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:41.057 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-25T11:20:41.057 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:41.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:41.057 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:41.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:41.114 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=foo bs=1M count=32) 2024-04-25T11:20:41.356 INFO:teuthology.orchestra.run.smithi106.stderr:32+0 records in 2024-04-25T11:20:41.357 INFO:teuthology.orchestra.run.smithi106.stderr:32+0 records out 2024-04-25T11:20:41.357 INFO:teuthology.orchestra.run.smithi106.stderr:33554432 bytes (34 MB, 32 MiB) copied, 0.185838 s, 181 MB/s 2024-04-25T11:20:41.358 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:41.358 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:41.365 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:41.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45d170640 1 Processor -- start 2024-04-25T11:20:41.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45d170640 1 -- start start 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45d170640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc45815d6f0 0x7fc45815db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45d170640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4580b5cc0 con 0x7fc4580b43f0 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45d170640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4580b5f80 con 0x7fc458160f10 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45d170640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc4580b6260 con 0x7fc45815d6f0 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45759e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc4580b43f0 0x7fc4580b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52616/0 (socket says 172.21.15.106:52616) 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45759e640 1 -- 172.21.15.106:0/2875987292 learned_addr learned my addr 172.21.15.106:0/2875987292 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45659c640 1 --2- 172.21.15.106:0/2875987292 >> v2:172.21.15.134:3300/0 conn(0x7fc45815d6f0 0x7fc45815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609844267 0 0) 0x7fc4580b5cc0 con 0x7fc4580b43f0 2024-04-25T11:20:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc440005740 con 0x7fc4580b43f0 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2039109290 0 0) 0x7fc440005740 con 0x7fc4580b43f0 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 >> v2:172.21.15.134:3300/0 conn(0x7fc45815d6f0 msgr2=0x7fc45815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 --2- 172.21.15.106:0/2875987292 >> v2:172.21.15.134:3300/0 conn(0x7fc45815d6f0 0x7fc45815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 >> v1:172.21.15.134:6789/0 conn(0x7fc458160f10 legacy=0x7fc45815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4580b6540 con 0x7fc4580b43f0 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fc450001a00 con 0x7fc4580b43f0 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 >> v1:172.21.15.106:6789/0 conn(0x7fc4580b43f0 legacy=0x7fc4580b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc440005bd0 0x7fc440006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440006a20 0x7fc440006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc440007470 0x7fc44000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc4580b6260 con 0x7fc440007470 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc44000e400 con 0x7fc440005bd0 2024-04-25T11:20:41.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc44000e600 con 0x7fc440006a20 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45759e640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc440007470 0x7fc44000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.563+0000 7fc45659c640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440006a20 0x7fc440006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc456d9d640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc440005bd0 0x7fc440006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45759e640 1 -- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440006a20 msgr2=0x7fc440006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45759e640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440006a20 0x7fc440006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45759e640 1 -- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc440005bd0 msgr2=0x7fc440006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45759e640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc440005bd0 0x7fc440006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45759e640 1 -- 172.21.15.106:0/2875987292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc440005740 con 0x7fc440007470 2024-04-25T11:20:41.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc456d9d640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc440005bd0 0x7fc440006040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:41.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45759e640 1 -- 172.21.15.106:0/2875987292 reap_dead start 2024-04-25T11:20:41.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45759e640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc440007470 0x7fc44000dbe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc45000a810 tx=0x7fc45001faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5787b422be695b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45000a460 con 0x7fc440007470 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4500aa6f0 con 0x7fc440007470 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc455d9b640 1 -- 172.21.15.106:0/2875987292 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4500aaae0 con 0x7fc440007470 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45d170640 1 -- 172.21.15.106:0/2875987292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc440007470 msgr2=0x7fc44000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45d170640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc440007470 0x7fc44000dbe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc45000a810 tx=0x7fc45001faa0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45d170640 1 -- 172.21.15.106:0/2875987292 shutdown_connections 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45d170640 1 --2- 172.21.15.106:0/2875987292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc440007470 0x7fc44000dbe0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45d170640 1 -- 172.21.15.106:0/2875987292 >> 172.21.15.106:0/2875987292 conn(0x7fc45806c7c0 msgr2=0x7fc45806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45d170640 1 -- 172.21.15.106:0/2875987292 shutdown_connections 2024-04-25T11:20:41.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.567+0000 7fc45d170640 1 -- 172.21.15.106:0/2875987292 wait complete. 2024-04-25T11:20:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45d170640 1 Processor -- start 2024-04-25T11:20:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45d170640 1 -- start start 2024-04-25T11:20:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45d170640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440007470 0x7fc4581561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45d170640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc45814ca10 0x7fc4580b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45d170640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580b3410 0x7fc4580ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45d170640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc45815db00 con 0x7fc4580b3410 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45d170640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc458161320 con 0x7fc45814ca10 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45d170640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc45815dc80 con 0x7fc440007470 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580b3410 0x7fc4580ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580b3410 0x7fc4580ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60440/0 (socket says 172.21.15.106:60440) 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 -- 172.21.15.106:0/3006127247 learned_addr learned my addr 172.21.15.106:0/3006127247 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45659c640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc45814ca10 0x7fc4580b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc456d9d640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440007470 0x7fc4581561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 -- 172.21.15.106:0/3006127247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440007470 msgr2=0x7fc4581561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440007470 0x7fc4581561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 -- 172.21.15.106:0/3006127247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc45814ca10 msgr2=0x7fc4580b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc45814ca10 0x7fc4580b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 -- 172.21.15.106:0/3006127247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4580d0480 con 0x7fc4580b3410 2024-04-25T11:20:41.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc45759e640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580b3410 0x7fc4580ae8a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc450020340 tx=0x7fc440007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc43f7fe640 1 -- 172.21.15.106:0/3006127247 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450001260 con 0x7fc4580b3410 2024-04-25T11:20:41.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc43f7fe640 1 -- 172.21.15.106:0/3006127247 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4500bb070 con 0x7fc4580b3410 2024-04-25T11:20:41.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.571+0000 7fc43f7fe640 1 -- 172.21.15.106:0/3006127247 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4500aaaa0 con 0x7fc4580b3410 2024-04-25T11:20:41.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.575+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4580b3aa0 con 0x7fc4580b3410 2024-04-25T11:20:41.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.575+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4580b3d10 con 0x7fc4580b3410 2024-04-25T11:20:41.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.575+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4580b1850 con 0x7fc4580b3410 2024-04-25T11:20:41.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.575+0000 7fc43f7fe640 1 -- 172.21.15.106:0/3006127247 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc4500aac40 con 0x7fc4580b3410 2024-04-25T11:20:41.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.575+0000 7fc43f7fe640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc44406f5d0 0x7fc444071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.575+0000 7fc43f7fe640 1 -- 172.21.15.106:0/3006127247 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc45012c9d0 con 0x7fc4580b3410 2024-04-25T11:20:41.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.575+0000 7fc456d9d640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc44406f5d0 0x7fc444071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.575+0000 7fc456d9d640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc44406f5d0 0x7fc444071ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc448007fd0 tx=0x7fc448009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.579+0000 7fc43f7fe640 1 -- 172.21.15.106:0/3006127247 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4500197b0 con 0x7fc4580b3410 2024-04-25T11:20:41.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.743+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc45814d440 con 0x7fc4580b3410 2024-04-25T11:20:41.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.743+0000 7fc43f7fe640 1 -- 172.21.15.106:0/3006127247 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7fc4500f98c0 con 0x7fc4580b3410 2024-04-25T11:20:41.744 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:41.744 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:33.406557+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:41.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc44406f5d0 msgr2=0x7fc444071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc44406f5d0 0x7fc444071ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc448007fd0 tx=0x7fc448009040 comp rx=0 tx=0).stop 2024-04-25T11:20:41.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580b3410 msgr2=0x7fc4580ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580b3410 0x7fc4580ae8a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc450020340 tx=0x7fc440007a50 comp rx=0 tx=0).stop 2024-04-25T11:20:41.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 shutdown_connections 2024-04-25T11:20:41.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fc44406f5d0 0x7fc444071ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580b3410 0x7fc4580ae8a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc45814ca10 0x7fc4580b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 --2- 172.21.15.106:0/3006127247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc440007470 0x7fc4581561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 >> 172.21.15.106:0/3006127247 conn(0x7fc45806c7c0 msgr2=0x7fc4580cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:41.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 shutdown_connections 2024-04-25T11:20:41.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.747+0000 7fc45d170640 1 -- 172.21.15.106:0/3006127247 wait complete. 2024-04-25T11:20:41.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:20:29.516176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:20:30.780121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:41.765 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:41.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.919+0000 7fdb9977e640 1 Processor -- start 2024-04-25T11:20:41.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.919+0000 7fdb9977e640 1 -- start start 2024-04-25T11:20:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb9977e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 0x7fdb9406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb9977e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdb940b0ac0 0x7fdb940b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb9977e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9406df50 con 0x7fdb940d08d0 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb9977e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdb940b1710 con 0x7fdb940b0ac0 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb9977e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9415def0 con 0x7fdb9406d600 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb937fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdb940d08d0 0x7fdb940d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52628/0 (socket says 172.21.15.106:52628) 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb937fe640 1 -- 172.21.15.106:0/439266468 learned_addr learned my addr 172.21.15.106:0/439266468 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb927fc640 1 --2- 172.21.15.106:0/439266468 >> v2:172.21.15.134:3300/0 conn(0x7fdb940b0ac0 0x7fdb940b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb92ffd640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 0x7fdb9406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385317677 0 0) 0x7fdb8c001120 con 0x7fdb940d08d0 2024-04-25T11:20:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb7c003530 con 0x7fdb940d08d0 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3816696828 0 0) 0x7fdb7c003530 con 0x7fdb940d08d0 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 msgr2=0x7fdb9406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 0x7fdb9406da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 >> v2:172.21.15.134:3300/0 conn(0x7fdb940b0ac0 msgr2=0x7fdb940b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 --2- 172.21.15.106:0/439266468 >> v2:172.21.15.134:3300/0 conn(0x7fdb940b0ac0 0x7fdb940b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9415d170 con 0x7fdb940d08d0 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fdb8c001850 con 0x7fdb940d08d0 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 >> v1:172.21.15.106:6789/0 conn(0x7fdb940d08d0 legacy=0x7fdb940d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0039c0 0x7fdb7c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c004810 0x7fdb7c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 0x7fdb7c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb940b1710 con 0x7fdb7c004810 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9415def0 con 0x7fdb7c0039c0 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7c00b9d0 con 0x7fdb9406d600 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb927fc640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c004810 0x7fdb7c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb937fe640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 0x7fdb7c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb92ffd640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0039c0 0x7fdb7c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb927fc640 1 -- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 msgr2=0x7fdb7c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb927fc640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 0x7fdb7c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb927fc640 1 -- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0039c0 msgr2=0x7fdb7c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb927fc640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0039c0 0x7fdb7c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb927fc640 1 -- 172.21.15.106:0/439266468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c003530 con 0x7fdb7c004810 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb92ffd640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb7c0039c0 0x7fdb7c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.923+0000 7fdb937fe640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb9406d600 0x7fdb7c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb927fc640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c004810 0x7fdb7c004c20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406fec0 tx=0x7fdb8801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242c2fc09135f78c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880096b0 con 0x7fdb7c004810 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb880a64e0 con 0x7fdb7c004810 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb91ffb640 1 -- 172.21.15.106:0/439266468 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880a68d0 con 0x7fdb7c004810 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb9977e640 1 -- 172.21.15.106:0/439266468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c004810 msgr2=0x7fdb7c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb9977e640 1 --2- 172.21.15.106:0/439266468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb7c004810 0x7fdb7c004c20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406fec0 tx=0x7fdb8801c9f0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb937fe640 1 -- 172.21.15.106:0/439266468 reap_dead start 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb9977e640 1 -- 172.21.15.106:0/439266468 shutdown_connections 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb9977e640 1 -- 172.21.15.106:0/439266468 >> 172.21.15.106:0/439266468 conn(0x7fdb94073300 msgr2=0x7fdb94073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb9977e640 1 -- 172.21.15.106:0/439266468 shutdown_connections 2024-04-25T11:20:41.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb9977e640 1 -- 172.21.15.106:0/439266468 wait complete. 2024-04-25T11:20:41.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb9977e640 1 Processor -- start 2024-04-25T11:20:41.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.927+0000 7fdb9977e640 1 -- start start 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb94141ee0 0x7fdb941e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940b7570 0x7fdb94141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb941417a0 0x7fdb9413f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9415da60 con 0x7fdb94141ee0 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94161450 con 0x7fdb940b7570 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9415d6f0 con 0x7fdb941417a0 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb94141ee0 0x7fdb941e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb94141ee0 0x7fdb941e78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60456/0 (socket says 172.21.15.106:60456) 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 -- 172.21.15.106:0/1542294731 learned_addr learned my addr 172.21.15.106:0/1542294731 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb927fc640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940b7570 0x7fdb94141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb937fe640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb941417a0 0x7fdb9413f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 -- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb941417a0 msgr2=0x7fdb9413f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb941417a0 0x7fdb9413f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 -- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940b7570 msgr2=0x7fdb94141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940b7570 0x7fdb94141260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 -- 172.21.15.106:0/1542294731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb941573a0 con 0x7fdb94141ee0 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb927fc640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940b7570 0x7fdb94141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb92ffd640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb94141ee0 0x7fdb941e78c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400f3b0 tx=0x7fdb8400f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb7b7fe640 1 -- 172.21.15.106:0/1542294731 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84003ba0 con 0x7fdb94141ee0 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb7b7fe640 1 -- 172.21.15.106:0/1542294731 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb84003d40 con 0x7fdb94141ee0 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb7b7fe640 1 -- 172.21.15.106:0/1542294731 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84004240 con 0x7fdb94141ee0 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb94141bb0 con 0x7fdb94141ee0 2024-04-25T11:20:41.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941577a0 con 0x7fdb94141ee0 2024-04-25T11:20:41.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb94050750 con 0x7fdb94141ee0 2024-04-25T11:20:41.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb7b7fe640 1 -- 172.21.15.106:0/1542294731 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb8401e020 con 0x7fdb94141ee0 2024-04-25T11:20:41.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb7b7fe640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb8006f430 0x7fdb80071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb7b7fe640 1 -- 172.21.15.106:0/1542294731 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdb84090030 con 0x7fdb94141ee0 2024-04-25T11:20:41.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.931+0000 7fdb927fc640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb8006f430 0x7fdb80071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.935+0000 7fdb927fc640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb8006f430 0x7fdb80071940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb880af000 tx=0x7fdb88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:41.935+0000 7fdb7b7fe640 1 -- 172.21.15.106:0/1542294731 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb84025070 con 0x7fdb94141ee0 2024-04-25T11:20:42.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb94074020 con 0x7fdb94141ee0 2024-04-25T11:20:42.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb7b7fe640 1 -- 172.21.15.106:0/1542294731 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7fdb94141bb0 con 0x7fdb94141ee0 2024-04-25T11:20:42.099 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb8006f430 msgr2=0x7fdb80071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb8006f430 0x7fdb80071940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb880af000 tx=0x7fdb88008040 comp rx=0 tx=0).stop 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb94141ee0 msgr2=0x7fdb941e78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb94141ee0 0x7fdb941e78c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400f3b0 tx=0x7fdb8400f870 comp rx=0 tx=0).stop 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 shutdown_connections 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb941417a0 0x7fdb9413f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940b7570 0x7fdb94141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdb8006f430 0x7fdb80071940 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 --2- 172.21.15.106:0/1542294731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb94141ee0 0x7fdb941e78c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.099+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 >> 172.21.15.106:0/1542294731 conn(0x7fdb94073300 msgr2=0x7fdb940707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.103+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 shutdown_connections 2024-04-25T11:20:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.103+0000 7fdb9977e640 1 -- 172.21.15.106:0/1542294731 wait complete. 2024-04-25T11:20:42.120 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:42.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38c2a10640 1 Processor -- start 2024-04-25T11:20:42.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38c2a10640 1 -- start start 2024-04-25T11:20:42.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38c2a10640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38bc0cf4d0 0x7f38bc0d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38c2a10640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38bc0d0cb0 0x7f38bc0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38c2a10640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc06ec70 con 0x7f38bc15ca10 2024-04-25T11:20:42.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38c2a10640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc0707b0 con 0x7f38bc0cf4d0 2024-04-25T11:20:42.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38c2a10640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc06f880 con 0x7f38bc0d0cb0 2024-04-25T11:20:42.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38bbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38bc0cf4d0 0x7f38bc0d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38bbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38bc0cf4d0 0x7f38bc0d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60472/0 (socket says 172.21.15.106:60472) 2024-04-25T11:20:42.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.275+0000 7f38bbfff640 1 -- 172.21.15.106:0/2070506949 learned_addr learned my addr 172.21.15.106:0/2070506949 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:42.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38bb7fe640 1 --2- 172.21.15.106:0/2070506949 >> v2:172.21.15.134:3300/0 conn(0x7f38bc0d0cb0 0x7f38bc0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38bbfff640 1 -- 172.21.15.106:0/2070506949 >> v2:172.21.15.134:3300/0 conn(0x7f38bc0d0cb0 msgr2=0x7f38bc0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:42.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38bbfff640 1 --2- 172.21.15.106:0/2070506949 >> v2:172.21.15.134:3300/0 conn(0x7f38bc0d0cb0 0x7f38bc0d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38bbfff640 1 -- 172.21.15.106:0/2070506949 >> v1:172.21.15.106:6789/0 conn(0x7f38bc15ca10 legacy=0x7f38bc0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38bbfff640 1 -- 172.21.15.106:0/2070506949 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc06ef50 con 0x7f38bc0cf4d0 2024-04-25T11:20:42.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38baffd640 1 -- 172.21.15.106:0/2070506949 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361528593 0 0) 0x7f38bc06ec70 con 0x7f38bc15ca10 2024-04-25T11:20:42.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38bbfff640 1 --2- 172.21.15.106:0/2070506949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc0cf4d0 0x7f38bc0d1740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f38bc160410 tx=0x7f38ac01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1352832bcef9c431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38baffd640 1 -- 172.21.15.106:0/2070506949 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0b4070 con 0x7f38bc0cf4d0 2024-04-25T11:20:42.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38baffd640 1 -- 172.21.15.106:0/2070506949 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac0a7a30 con 0x7f38bc0cf4d0 2024-04-25T11:20:42.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38baffd640 1 -- 172.21.15.106:0/2070506949 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0a7e00 con 0x7f38bc0cf4d0 2024-04-25T11:20:42.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38c2a10640 1 -- 172.21.15.106:0/2070506949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc0cf4d0 msgr2=0x7f38bc0d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38c2a10640 1 --2- 172.21.15.106:0/2070506949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc0cf4d0 0x7f38bc0d1740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f38bc160410 tx=0x7f38ac01eb60 comp rx=0 tx=0).stop 2024-04-25T11:20:42.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38c2a10640 1 -- 172.21.15.106:0/2070506949 shutdown_connections 2024-04-25T11:20:42.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38c2a10640 1 --2- 172.21.15.106:0/2070506949 >> v2:172.21.15.134:3300/0 conn(0x7f38bc0d0cb0 0x7f38bc0d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38c2a10640 1 --2- 172.21.15.106:0/2070506949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc0cf4d0 0x7f38bc0d1740 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38c2a10640 1 -- 172.21.15.106:0/2070506949 >> 172.21.15.106:0/2070506949 conn(0x7f38bc06c9f0 msgr2=0x7f38bc06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:42.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38c2a10640 1 -- 172.21.15.106:0/2070506949 shutdown_connections 2024-04-25T11:20:42.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.279+0000 7f38c2a10640 1 -- 172.21.15.106:0/2070506949 wait complete. 2024-04-25T11:20:42.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 Processor -- start 2024-04-25T11:20:42.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 -- start start 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38bc0cf4d0 0x7f38bc0ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38bc0d0cb0 0x7f38bc15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc15ca10 0x7f38bc14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc072db0 con 0x7f38bc15ca10 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc072c30 con 0x7f38bc0d0cb0 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc072f30 con 0x7f38bc0cf4d0 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc15ca10 0x7f38bc14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc15ca10 0x7f38bc14bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60476/0 (socket says 172.21.15.106:60476) 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 -- 172.21.15.106:0/3806561445 learned_addr learned my addr 172.21.15.106:0/3806561445 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38bbfff640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38bc0cf4d0 0x7f38bc0ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38bb7fe640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38bc0d0cb0 0x7f38bc15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 -- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38bc0cf4d0 msgr2=0x7f38bc0ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38bc0cf4d0 0x7f38bc0ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 -- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38bc0d0cb0 msgr2=0x7f38bc15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38bc0d0cb0 0x7f38bc15bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 -- 172.21.15.106:0/3806561445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc0aa530 con 0x7f38bc15ca10 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38bb7fe640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38bc0d0cb0 0x7f38bc15bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c0fad640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc15ca10 0x7f38bc14bce0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f38b400c780 tx=0x7f38b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38b8ff9640 1 -- 172.21.15.106:0/3806561445 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4003db0 con 0x7f38bc15ca10 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38b8ff9640 1 -- 172.21.15.106:0/3806561445 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b400ce30 con 0x7f38bc15ca10 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc0aa810 con 0x7f38bc15ca10 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38b8ff9640 1 -- 172.21.15.106:0/3806561445 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4005290 con 0x7f38bc15ca10 2024-04-25T11:20:42.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.283+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc0aadd0 con 0x7f38bc15ca10 2024-04-25T11:20:42.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.287+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc050750 con 0x7f38bc15ca10 2024-04-25T11:20:42.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.287+0000 7f38b8ff9640 1 -- 172.21.15.106:0/3806561445 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f38b4005430 con 0x7f38bc15ca10 2024-04-25T11:20:42.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.287+0000 7f38b8ff9640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38a806f400 0x7f38a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.287+0000 7f38bbfff640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38a806f400 0x7f38a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.287+0000 7f38b8ff9640 1 -- 172.21.15.106:0/3806561445 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f38b408f1a0 con 0x7f38bc15ca10 2024-04-25T11:20:42.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.287+0000 7f38bbfff640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38a806f400 0x7f38a8071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0b1000 tx=0x7f38ac0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.291+0000 7f38b8ff9640 1 -- 172.21.15.106:0/3806561445 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38b405c090 con 0x7f38bc15ca10 2024-04-25T11:20:42.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.447+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f38bc15c330 con 0x7f38bc15ca10 2024-04-25T11:20:42.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.447+0000 7f38b8ff9640 1 -- 172.21.15.106:0/3806561445 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f38b405b990 con 0x7f38bc15ca10 2024-04-25T11:20:42.448 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:42.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38a806f400 msgr2=0x7f38a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38a806f400 0x7f38a8071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0b1000 tx=0x7f38ac0a9070 comp rx=0 tx=0).stop 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc15ca10 msgr2=0x7f38bc14bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc15ca10 0x7f38bc14bce0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f38b400c780 tx=0x7f38b400cc40 comp rx=0 tx=0).stop 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 shutdown_connections 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38a806f400 0x7f38a8071910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38bc15ca10 0x7f38bc14bce0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38bc0d0cb0 0x7f38bc15bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 --2- 172.21.15.106:0/3806561445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38bc0cf4d0 0x7f38bc0ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 >> 172.21.15.106:0/3806561445 conn(0x7f38bc06c9f0 msgr2=0x7f38bc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 shutdown_connections 2024-04-25T11:20:42.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.451+0000 7f38c2a10640 1 -- 172.21.15.106:0/3806561445 wait complete. 2024-04-25T11:20:42.467 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:42.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43cf195640 1 Processor -- start 2024-04-25T11:20:42.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43cf195640 1 -- start start 2024-04-25T11:20:42.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43cf195640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43c80b7cb0 0x7f43c80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43cf195640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cf340 0x7f43c80b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43cf195640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c80b4d70 con 0x7f43c80cef30 2024-04-25T11:20:42.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43cf195640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f43c80cdb20 con 0x7f43c80b7cb0 2024-04-25T11:20:42.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43cf195640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f43c80b3c70 con 0x7f43c80cf340 2024-04-25T11:20:42.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43ccf31640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43c80b7cb0 0x7f43c80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43ccf31640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43c80b7cb0 0x7f43c80cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60488/0 (socket says 172.21.15.106:60488) 2024-04-25T11:20:42.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43ccf31640 1 -- 172.21.15.106:0/453451926 learned_addr learned my addr 172.21.15.106:0/453451926 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:42.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43cd732640 1 --2- 172.21.15.106:0/453451926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cf340 0x7f43c80b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43ccf31640 1 -- 172.21.15.106:0/453451926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cf340 msgr2=0x7f43c80b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:42.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43ccf31640 1 --2- 172.21.15.106:0/453451926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cf340 0x7f43c80b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43ccf31640 1 -- 172.21.15.106:0/453451926 >> v1:172.21.15.134:6789/0 conn(0x7f43c80cef30 legacy=0x7f43c80b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:42.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.623+0000 7f43ccf31640 1 -- 172.21.15.106:0/453451926 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c80ce4f0 con 0x7f43c80b7cb0 2024-04-25T11:20:42.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43ccf31640 1 --2- 172.21.15.106:0/453451926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c80cdfb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009870 tx=0x7f43b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d22eb134a645445 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43c77fe640 1 -- 172.21.15.106:0/453451926 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80b0070 con 0x7f43c80b7cb0 2024-04-25T11:20:42.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43c77fe640 1 -- 172.21.15.106:0/453451926 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b80ab600 con 0x7f43c80b7cb0 2024-04-25T11:20:42.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43c77fe640 1 -- 172.21.15.106:0/453451926 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80ab9d0 con 0x7f43c80b7cb0 2024-04-25T11:20:42.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43cf195640 1 -- 172.21.15.106:0/453451926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 msgr2=0x7f43c80cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43cf195640 1 --2- 172.21.15.106:0/453451926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c80cdfb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009870 tx=0x7f43b801c520 comp rx=0 tx=0).stop 2024-04-25T11:20:42.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43cf195640 1 -- 172.21.15.106:0/453451926 shutdown_connections 2024-04-25T11:20:42.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43cf195640 1 --2- 172.21.15.106:0/453451926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cf340 0x7f43c80b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43cf195640 1 --2- 172.21.15.106:0/453451926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c80cdfb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43cf195640 1 -- 172.21.15.106:0/453451926 >> 172.21.15.106:0/453451926 conn(0x7f43c80b40b0 msgr2=0x7f43c80b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:42.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43cf195640 1 -- 172.21.15.106:0/453451926 shutdown_connections 2024-04-25T11:20:42.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.627+0000 7f43cf195640 1 -- 172.21.15.106:0/453451926 wait complete. 2024-04-25T11:20:42.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cf195640 1 Processor -- start 2024-04-25T11:20:42.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cf195640 1 -- start start 2024-04-25T11:20:42.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cf195640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c8155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cf195640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cef30 0x7f43c806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cf195640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c80cf340 0x7f43c80cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cf195640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80d0f10 con 0x7f43c80b7cb0 2024-04-25T11:20:42.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cf195640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80d0d90 con 0x7f43c80cf340 2024-04-25T11:20:42.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cf195640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f43c80d1090 con 0x7f43c80cef30 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c8155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c8155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60496/0 (socket says 172.21.15.106:60496) 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 -- 172.21.15.106:0/2435198620 learned_addr learned my addr 172.21.15.106:0/2435198620 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43cd732640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c80cf340 0x7f43c80cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43c7fff640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cef30 0x7f43c806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 -- 172.21.15.106:0/2435198620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cef30 msgr2=0x7f43c806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cef30 0x7f43c806cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 -- 172.21.15.106:0/2435198620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c80cf340 msgr2=0x7f43c80cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c80cf340 0x7f43c80cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 -- 172.21.15.106:0/2435198620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c809c810 con 0x7f43c80b7cb0 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.631+0000 7f43ccf31640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c8155b20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f43b80ae040 tx=0x7f43b801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43c57fa640 1 -- 172.21.15.106:0/2435198620 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80b0040 con 0x7f43c80b7cb0 2024-04-25T11:20:42.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43c57fa640 1 -- 172.21.15.106:0/2435198620 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b80bc070 con 0x7f43c80b7cb0 2024-04-25T11:20:42.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c809caf0 con 0x7f43c80b7cb0 2024-04-25T11:20:42.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43c57fa640 1 -- 172.21.15.106:0/2435198620 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80aac30 con 0x7f43c80b7cb0 2024-04-25T11:20:42.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c809d0b0 con 0x7f43c80b7cb0 2024-04-25T11:20:42.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c8050750 con 0x7f43c80b7cb0 2024-04-25T11:20:42.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43c57fa640 1 -- 172.21.15.106:0/2435198620 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43b80c1460 con 0x7f43c80b7cb0 2024-04-25T11:20:42.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43c57fa640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43b406f400 0x7f43b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43c7fff640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43b406f400 0x7f43b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43c57fa640 1 -- 172.21.15.106:0/2435198620 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f43b8129b50 con 0x7f43c80b7cb0 2024-04-25T11:20:42.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.635+0000 7f43c7fff640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43b406f400 0x7f43b4071910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43c81faa00 tx=0x7f43bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.639+0000 7f43c57fa640 1 -- 172.21.15.106:0/2435198620 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f43b8016110 con 0x7f43c80b7cb0 2024-04-25T11:20:42.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.799+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43c80c5dd0 con 0x7f43c80b7cb0 2024-04-25T11:20:42.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43c57fa640 1 -- 172.21.15.106:0/2435198620 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f43b80f69a0 con 0x7f43c80b7cb0 2024-04-25T11:20:42.801 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43b406f400 msgr2=0x7f43b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43b406f400 0x7f43b4071910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43c81faa00 tx=0x7f43bc008040 comp rx=0 tx=0).stop 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 msgr2=0x7f43c8155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c8155b20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f43b80ae040 tx=0x7f43b801ad60 comp rx=0 tx=0).stop 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 shutdown_connections 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c80cf340 0x7f43c80cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43c80cef30 0x7f43c806cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f43b406f400 0x7f43b4071910 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 --2- 172.21.15.106:0/2435198620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43c80b7cb0 0x7f43c8155b20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 >> 172.21.15.106:0/2435198620 conn(0x7f43c80b40b0 msgr2=0x7f43c80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:42.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 shutdown_connections 2024-04-25T11:20:42.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.803+0000 7f43cf195640 1 -- 172.21.15.106:0/2435198620 wait complete. 2024-04-25T11:20:42.822 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:42.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.967+0000 7f1729701640 1 Processor -- start 2024-04-25T11:20:42.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.967+0000 7f1729701640 1 -- start start 2024-04-25T11:20:42.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1729701640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f172415cae0 0x7f1724160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1729701640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 0x7f17240cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1729701640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17240b6120 con 0x7f17240b4bd0 2024-04-25T11:20:42.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1729701640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f17240d0020 con 0x7f172415cae0 2024-04-25T11:20:42.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1729701640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17240ce2a0 con 0x7f17240cf6d0 2024-04-25T11:20:42.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1722ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f172415cae0 0x7f1724160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1722ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f172415cae0 0x7f1724160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60502/0 (socket says 172.21.15.106:60502) 2024-04-25T11:20:42.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1722ffd640 1 -- 172.21.15.106:0/1442682568 learned_addr learned my addr 172.21.15.106:0/1442682568 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:42.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f171bfff640 1 --2- 172.21.15.106:0/1442682568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 0x7f17240cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1722ffd640 1 -- 172.21.15.106:0/1442682568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 msgr2=0x7f17240cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:42.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1722ffd640 1 --2- 172.21.15.106:0/1442682568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 0x7f17240cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1722ffd640 1 -- 172.21.15.106:0/1442682568 >> v1:172.21.15.106:6789/0 conn(0x7f17240b4bd0 legacy=0x7f17240b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f17227fc640 1 -- 172.21.15.106:0/1442682568 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 928743897 0 0) 0x7f17240b6120 con 0x7f17240b4bd0 2024-04-25T11:20:42.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1722ffd640 1 -- 172.21.15.106:0/1442682568 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1724070920 con 0x7f172415cae0 2024-04-25T11:20:42.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f1722ffd640 1 --2- 172.21.15.106:0/1442682568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172415cae0 0x7f1724160fc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f170c00bb10 tx=0x7f170c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca848a0e5d2d45e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f17227fc640 1 -- 172.21.15.106:0/1442682568 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c0b4070 con 0x7f172415cae0 2024-04-25T11:20:42.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f17227fc640 1 -- 172.21.15.106:0/1442682568 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170c0a7a70 con 0x7f172415cae0 2024-04-25T11:20:42.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.971+0000 7f17227fc640 1 -- 172.21.15.106:0/1442682568 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c0a7e40 con 0x7f172415cae0 2024-04-25T11:20:42.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- 172.21.15.106:0/1442682568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172415cae0 msgr2=0x7f1724160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 --2- 172.21.15.106:0/1442682568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172415cae0 0x7f1724160fc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f170c00bb10 tx=0x7f170c01eba0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- 172.21.15.106:0/1442682568 shutdown_connections 2024-04-25T11:20:42.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 --2- 172.21.15.106:0/1442682568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 0x7f17240cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 --2- 172.21.15.106:0/1442682568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172415cae0 0x7f1724160fc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- 172.21.15.106:0/1442682568 >> 172.21.15.106:0/1442682568 conn(0x7f17240b3560 msgr2=0x7f17240b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:42.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- 172.21.15.106:0/1442682568 shutdown_connections 2024-04-25T11:20:42.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- 172.21.15.106:0/1442682568 wait complete. 2024-04-25T11:20:42.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 Processor -- start 2024-04-25T11:20:42.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- start start 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17240b4bd0 0x7f1724151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 0x7f172414df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172415cae0 0x7f172414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17240736a0 con 0x7f17240b4bd0 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1724073520 con 0x7f172415cae0 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1729701640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1724073820 con 0x7f17240cf6d0 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1722ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17240b4bd0 0x7f1724151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1722ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17240b4bd0 0x7f1724151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60514/0 (socket says 172.21.15.106:60514) 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.975+0000 7f1722ffd640 1 -- 172.21.15.106:0/86593896 learned_addr learned my addr 172.21.15.106:0/86593896 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171bfff640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 0x7f172414df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f17237fe640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172415cae0 0x7f172414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1722ffd640 1 -- 172.21.15.106:0/86593896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 msgr2=0x7f172414df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1722ffd640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 0x7f172414df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1722ffd640 1 -- 172.21.15.106:0/86593896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172415cae0 msgr2=0x7f172414e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1722ffd640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172415cae0 0x7f172414e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1722ffd640 1 -- 172.21.15.106:0/86593896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1724151fc0 con 0x7f17240b4bd0 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1722ffd640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17240b4bd0 0x7f1724151520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f170c00bb10 tx=0x7f170c01b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171b7fe640 1 -- 172.21.15.106:0/86593896 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c0b4040 con 0x7f17240b4bd0 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171b7fe640 1 -- 172.21.15.106:0/86593896 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170c004030 con 0x7f17240b4bd0 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1724159a30 con 0x7f17240b4bd0 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171b7fe640 1 -- 172.21.15.106:0/86593896 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c004420 con 0x7f17240b4bd0 2024-04-25T11:20:42.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1724159ff0 con 0x7f17240b4bd0 2024-04-25T11:20:42.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1724050750 con 0x7f17240b4bd0 2024-04-25T11:20:42.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171b7fe640 1 -- 172.21.15.106:0/86593896 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f170c0b7070 con 0x7f17240b4bd0 2024-04-25T11:20:42.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171b7fe640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f170406f430 0x7f1704071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171bfff640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f170406f430 0x7f1704071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171b7fe640 1 -- 172.21.15.106:0/86593896 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f170c12c5e0 con 0x7f17240b4bd0 2024-04-25T11:20:42.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.979+0000 7f171bfff640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f170406f430 0x7f1704071940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1714003fd0 tx=0x7f17140039c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:42.983+0000 7f171b7fe640 1 -- 172.21.15.106:0/86593896 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f170c0be020 con 0x7f17240b4bd0 2024-04-25T11:20:43.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.155+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f17240b0db0 con 0x7f17240b4bd0 2024-04-25T11:20:43.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.155+0000 7f171b7fe640 1 -- 172.21.15.106:0/86593896 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f170c018820 con 0x7f17240b4bd0 2024-04-25T11:20:43.156 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:43.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f170406f430 msgr2=0x7f1704071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f170406f430 0x7f1704071940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1714003fd0 tx=0x7f17140039c0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17240b4bd0 msgr2=0x7f1724151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17240b4bd0 0x7f1724151520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f170c00bb10 tx=0x7f170c01b940 comp rx=0 tx=0).stop 2024-04-25T11:20:43.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 shutdown_connections 2024-04-25T11:20:43.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172415cae0 0x7f172414e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17240cf6d0 0x7f172414df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f170406f430 0x7f1704071940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 --2- 172.21.15.106:0/86593896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17240b4bd0 0x7f1724151520 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 >> 172.21.15.106:0/86593896 conn(0x7f17240b3560 msgr2=0x7f17240b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:43.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 shutdown_connections 2024-04-25T11:20:43.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.159+0000 7f1729701640 1 -- 172.21.15.106:0/86593896 wait complete. 2024-04-25T11:20:43.177 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:43.177 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:43.182 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.355+0000 7f5ac49e5640 1 Processor -- start 2024-04-25T11:20:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5ac49e5640 1 -- start start 2024-04-25T11:20:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5ac49e5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5ac00cef60 0x7f5ac0073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5ac49e5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ac00d0ce0 0x7f5ac00d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5ac49e5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac00ce050 con 0x7f5ac015e0a0 2024-04-25T11:20:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5ac49e5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac015e6b0 con 0x7f5ac00d0ce0 2024-04-25T11:20:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5ac49e5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac015e990 con 0x7f5ac00cef60 2024-04-25T11:20:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5abed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ac00d0ce0 0x7f5ac00d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5abed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ac00d0ce0 0x7f5ac00d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60516/0 (socket says 172.21.15.106:60516) 2024-04-25T11:20:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5abed9d640 1 -- 172.21.15.106:0/695405636 learned_addr learned my addr 172.21.15.106:0/695405636 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5abdd9b640 1 --2- 172.21.15.106:0/695405636 >> v2:172.21.15.134:3300/0 conn(0x7f5ac00cef60 0x7f5ac0073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5abed9d640 1 -- 172.21.15.106:0/695405636 >> v2:172.21.15.134:3300/0 conn(0x7f5ac00cef60 msgr2=0x7f5ac0073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5abed9d640 1 --2- 172.21.15.106:0/695405636 >> v2:172.21.15.134:3300/0 conn(0x7f5ac00cef60 0x7f5ac0073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5abed9d640 1 -- 172.21.15.106:0/695405636 >> v1:172.21.15.134:6789/0 conn(0x7f5ac015e0a0 legacy=0x7f5ac00ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.359+0000 7f5abed9d640 1 -- 172.21.15.106:0/695405636 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac015ec70 con 0x7f5ac00d0ce0 2024-04-25T11:20:43.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5abed9d640 1 --2- 172.21.15.106:0/695405636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac00d0ce0 0x7f5ac00d10f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00b6850 tx=0x7f5ab801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c1e7446d679f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5abd59a640 1 -- 172.21.15.106:0/695405636 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80a5630 con 0x7f5ac00d0ce0 2024-04-25T11:20:43.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5abd59a640 1 -- 172.21.15.106:0/695405636 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab80a57d0 con 0x7f5ac00d0ce0 2024-04-25T11:20:43.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5abd59a640 1 -- 172.21.15.106:0/695405636 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80a5ba0 con 0x7f5ac00d0ce0 2024-04-25T11:20:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 -- 172.21.15.106:0/695405636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac00d0ce0 msgr2=0x7f5ac00d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/695405636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac00d0ce0 0x7f5ac00d10f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00b6850 tx=0x7f5ab801c970 comp rx=0 tx=0).stop 2024-04-25T11:20:43.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 -- 172.21.15.106:0/695405636 shutdown_connections 2024-04-25T11:20:43.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/695405636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac00d0ce0 0x7f5ac00d10f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/695405636 >> v2:172.21.15.134:3300/0 conn(0x7f5ac00cef60 0x7f5ac0073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 -- 172.21.15.106:0/695405636 >> 172.21.15.106:0/695405636 conn(0x7f5ac006c7c0 msgr2=0x7f5ac006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:43.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 -- 172.21.15.106:0/695405636 shutdown_connections 2024-04-25T11:20:43.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 -- 172.21.15.106:0/695405636 wait complete. 2024-04-25T11:20:43.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 Processor -- start 2024-04-25T11:20:43.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.363+0000 7f5ac49e5640 1 -- start start 2024-04-25T11:20:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ac00d0ce0 0x7f5ac0139700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ac015e0a0 0x7f5ac015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac015d4a0 0x7f5ac01456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac016a870 con 0x7f5ac015d4a0 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac016a6f0 con 0x7f5ac015e0a0 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac016a9f0 con 0x7f5ac00d0ce0 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac015d4a0 0x7f5ac01456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac015d4a0 0x7f5ac01456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60530/0 (socket says 172.21.15.106:60530) 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 -- 172.21.15.106:0/2340690123 learned_addr learned my addr 172.21.15.106:0/2340690123 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abe59c640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ac00d0ce0 0x7f5ac0139700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abdd9b640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ac015e0a0 0x7f5ac015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 -- 172.21.15.106:0/2340690123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ac00d0ce0 msgr2=0x7f5ac0139700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ac00d0ce0 0x7f5ac0139700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 -- 172.21.15.106:0/2340690123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ac015e0a0 msgr2=0x7f5ac015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ac015e0a0 0x7f5ac015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 -- 172.21.15.106:0/2340690123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac0138660 con 0x7f5ac015d4a0 2024-04-25T11:20:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abed9d640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac015d4a0 0x7f5ac01456a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80af040 tx=0x7f5ab8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ab6ffd640 1 -- 172.21.15.106:0/2340690123 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8009510 con 0x7f5ac015d4a0 2024-04-25T11:20:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ab6ffd640 1 -- 172.21.15.106:0/2340690123 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab80be070 con 0x7f5ac015d4a0 2024-04-25T11:20:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ab6ffd640 1 -- 172.21.15.106:0/2340690123 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80b88b0 con 0x7f5ac015d4a0 2024-04-25T11:20:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac0138940 con 0x7f5ac015d4a0 2024-04-25T11:20:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac0138f00 con 0x7f5ac015d4a0 2024-04-25T11:20:43.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac015bce0 con 0x7f5ac015d4a0 2024-04-25T11:20:43.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ab6ffd640 1 -- 172.21.15.106:0/2340690123 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ab80b8a50 con 0x7f5ac015d4a0 2024-04-25T11:20:43.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ab6ffd640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa006f400 0x7f5aa0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5abe59c640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa006f400 0x7f5aa0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.367+0000 7f5ab6ffd640 1 -- 172.21.15.106:0/2340690123 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5ab812a2c0 con 0x7f5ac015d4a0 2024-04-25T11:20:43.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.371+0000 7f5abe59c640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa006f400 0x7f5aa0071910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b810 tx=0x7f5ab0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.371+0000 7f5ab6ffd640 1 -- 172.21.15.106:0/2340690123 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ab8016560 con 0x7f5ac015d4a0 2024-04-25T11:20:43.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.523+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ac01383f0 con 0x7f5ac015d4a0 2024-04-25T11:20:43.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ab6ffd640 1 -- 172.21.15.106:0/2340690123 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7f5ab80f71a0 con 0x7f5ac015d4a0 2024-04-25T11:20:43.526 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:43.527 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:33.406557+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa006f400 msgr2=0x7f5aa0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa006f400 0x7f5aa0071910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b810 tx=0x7f5ab0008040 comp rx=0 tx=0).stop 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac015d4a0 msgr2=0x7f5ac01456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac015d4a0 0x7f5ac01456a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80af040 tx=0x7f5ab8019710 comp rx=0 tx=0).stop 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 shutdown_connections 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa006f400 0x7f5aa0071910 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ac015d4a0 0x7f5ac01456a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ac015e0a0 0x7f5ac015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 --2- 172.21.15.106:0/2340690123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ac00d0ce0 0x7f5ac0139700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 >> 172.21.15.106:0/2340690123 conn(0x7f5ac006c7c0 msgr2=0x7f5ac013a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:43.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 shutdown_connections 2024-04-25T11:20:43.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.527+0000 7f5ac49e5640 1 -- 172.21.15.106:0/2340690123 wait complete. 2024-04-25T11:20:43.546 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:20:29.516176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:20:30.780121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:43.546 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:43.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.699+0000 7f09bdd20640 1 Processor -- start 2024-04-25T11:20:43.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.699+0000 7f09bdd20640 1 -- start start 2024-04-25T11:20:43.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09bdd20640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09b815c070 0x7f09b8070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09bdd20640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b806ea20 0x7f09b806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09bdd20640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b80b4570 con 0x7f09b8072a20 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09bdd20640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f09b80b5180 con 0x7f09b815c070 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09bdd20640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09b80cf0c0 con 0x7f09b806ea20 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09b815c070 0x7f09b8070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b6ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f09b8072a20 0x7f09b8072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52654/0 (socket says 172.21.15.106:52654) 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b6ffd640 1 -- 172.21.15.106:0/3708528794 learned_addr learned my addr 172.21.15.106:0/3708528794 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b7fff640 1 --2- 172.21.15.106:0/3708528794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b806ea20 0x7f09b806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b77fe640 1 -- 172.21.15.106:0/3708528794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b806ea20 msgr2=0x7f09b806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b77fe640 1 --2- 172.21.15.106:0/3708528794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b806ea20 0x7f09b806ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b77fe640 1 -- 172.21.15.106:0/3708528794 >> v1:172.21.15.106:6789/0 conn(0x7f09b8072a20 legacy=0x7f09b8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b77fe640 1 -- 172.21.15.106:0/3708528794 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b80b4850 con 0x7f09b815c070 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.703+0000 7f09b67fc640 1 -- 172.21.15.106:0/3708528794 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076410074 0 0) 0x7f09b80b4570 con 0x7f09b8072a20 2024-04-25T11:20:43.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09b77fe640 1 --2- 172.21.15.106:0/3708528794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b815c070 0x7f09b8070730 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f09a800a960 tx=0x7f09a801d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4515bfbc19e720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09b67fc640 1 -- 172.21.15.106:0/3708528794 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80b3070 con 0x7f09b815c070 2024-04-25T11:20:43.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09b67fc640 1 -- 172.21.15.106:0/3708528794 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a80a6870 con 0x7f09b815c070 2024-04-25T11:20:43.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09b67fc640 1 -- 172.21.15.106:0/3708528794 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a800b040 con 0x7f09b815c070 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 -- 172.21.15.106:0/3708528794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b815c070 msgr2=0x7f09b8070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3708528794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b815c070 0x7f09b8070730 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f09a800a960 tx=0x7f09a801d840 comp rx=0 tx=0).stop 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 -- 172.21.15.106:0/3708528794 shutdown_connections 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3708528794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b806ea20 0x7f09b806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3708528794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b815c070 0x7f09b8070730 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 -- 172.21.15.106:0/3708528794 >> 172.21.15.106:0/3708528794 conn(0x7f09b806c9f0 msgr2=0x7f09b806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 -- 172.21.15.106:0/3708528794 shutdown_connections 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 -- 172.21.15.106:0/3708528794 wait complete. 2024-04-25T11:20:43.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 Processor -- start 2024-04-25T11:20:43.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.707+0000 7f09bdd20640 1 -- start start 2024-04-25T11:20:43.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b806ea20 0x7f09b814f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b8072a20 0x7f09b81565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b815c070 0x7f09b81f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09b806f780 con 0x7f09b8072a20 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8070c70 con 0x7f09b806ea20 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09b80b6220 con 0x7f09b815c070 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b8072a20 0x7f09b81565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b8072a20 0x7f09b81565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60552/0 (socket says 172.21.15.106:60552) 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 -- 172.21.15.106:0/3610893893 learned_addr learned my addr 172.21.15.106:0/3610893893 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b7fff640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b815c070 0x7f09b81f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b77fe640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b806ea20 0x7f09b814f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 -- 172.21.15.106:0/3610893893 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b815c070 msgr2=0x7f09b81f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b815c070 0x7f09b81f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 -- 172.21.15.106:0/3610893893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b806ea20 msgr2=0x7f09b814f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b806ea20 0x7f09b814f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 -- 172.21.15.106:0/3610893893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b81543b0 con 0x7f09b8072a20 2024-04-25T11:20:43.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b6ffd640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b8072a20 0x7f09b81565b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00c480 tx=0x7f09ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f099ffff640 1 -- 172.21.15.106:0/3610893893 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac018070 con 0x7f09b8072a20 2024-04-25T11:20:43.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f099ffff640 1 -- 172.21.15.106:0/3610893893 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09ac0074e0 con 0x7f09b8072a20 2024-04-25T11:20:43.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b81557f0 con 0x7f09b8072a20 2024-04-25T11:20:43.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f099ffff640 1 -- 172.21.15.106:0/3610893893 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac0049f0 con 0x7f09b8072a20 2024-04-25T11:20:43.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b8155db0 con 0x7f09b8072a20 2024-04-25T11:20:43.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b8050750 con 0x7f09b8072a20 2024-04-25T11:20:43.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f099ffff640 1 -- 172.21.15.106:0/3610893893 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f09ac01b070 con 0x7f09b8072a20 2024-04-25T11:20:43.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f099ffff640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09a406f430 0x7f09a4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f099ffff640 1 -- 172.21.15.106:0/3610893893 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f09ac08e630 con 0x7f09b8072a20 2024-04-25T11:20:43.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.711+0000 7f09b77fe640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09a406f430 0x7f09a4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.715+0000 7f09b77fe640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09a406f430 0x7f09a4071940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f09a80b0000 tx=0x7f09a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.719+0000 7f099ffff640 1 -- 172.21.15.106:0/3610893893 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09ac05b5b0 con 0x7f09b8072a20 2024-04-25T11:20:43.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.879+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09b80d0d70 con 0x7f09b8072a20 2024-04-25T11:20:43.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.879+0000 7f099ffff640 1 -- 172.21.15.106:0/3610893893 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f09ac05aeb0 con 0x7f09b8072a20 2024-04-25T11:20:43.880 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:43.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09a406f430 msgr2=0x7f09a4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09a406f430 0x7f09a4071940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f09a80b0000 tx=0x7f09a8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:43.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b8072a20 msgr2=0x7f09b81565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b8072a20 0x7f09b81565b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00c480 tx=0x7f09ac00c940 comp rx=0 tx=0).stop 2024-04-25T11:20:43.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 shutdown_connections 2024-04-25T11:20:43.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09b815c070 0x7f09b81f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f09a406f430 0x7f09a4071940 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09b8072a20 0x7f09b81565b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 --2- 172.21.15.106:0/3610893893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09b806ea20 0x7f09b814f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 >> 172.21.15.106:0/3610893893 conn(0x7f09b806c9f0 msgr2=0x7f09b815e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:43.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 shutdown_connections 2024-04-25T11:20:43.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:43.883+0000 7f09bdd20640 1 -- 172.21.15.106:0/3610893893 wait complete. 2024-04-25T11:20:43.900 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:44.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe7a20a640 1 Processor -- start 2024-04-25T11:20:44.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe7a20a640 1 -- start start 2024-04-25T11:20:44.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe7a20a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efe740cd930 0x7efe740d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe7a20a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe740d1c80 0x7efe740b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe7a20a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe740cdd40 con 0x7efe74143fa0 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe7a20a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efe740d2090 con 0x7efe740cd930 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe7a20a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe74073300 con 0x7efe740d1c80 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe72ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efe740cd930 0x7efe740d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe72ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efe740cd930 0x7efe740d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60558/0 (socket says 172.21.15.106:60558) 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe72ffd640 1 -- 172.21.15.106:0/2268402166 learned_addr learned my addr 172.21.15.106:0/2268402166 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe73fff640 1 --2- 172.21.15.106:0/2268402166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe740d1c80 0x7efe740b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe72ffd640 1 -- 172.21.15.106:0/2268402166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe740d1c80 msgr2=0x7efe740b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe72ffd640 1 --2- 172.21.15.106:0/2268402166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe740d1c80 0x7efe740b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe72ffd640 1 -- 172.21.15.106:0/2268402166 >> v1:172.21.15.134:6789/0 conn(0x7efe74143fa0 legacy=0x7efe74072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.043+0000 7efe72ffd640 1 -- 172.21.15.106:0/2268402166 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe740735e0 con 0x7efe740cd930 2024-04-25T11:20:44.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe72ffd640 1 --2- 172.21.15.106:0/2268402166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe740d1740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efe68009870 tx=0x7efe6801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d233f062d7098db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe727fc640 1 -- 172.21.15.106:0/2268402166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe680ab650 con 0x7efe740cd930 2024-04-25T11:20:44.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe727fc640 1 -- 172.21.15.106:0/2268402166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe680b0070 con 0x7efe740cd930 2024-04-25T11:20:44.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe727fc640 1 -- 172.21.15.106:0/2268402166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe680aba10 con 0x7efe740cd930 2024-04-25T11:20:44.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe7a20a640 1 -- 172.21.15.106:0/2268402166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 msgr2=0x7efe740d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe7a20a640 1 --2- 172.21.15.106:0/2268402166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe740d1740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efe68009870 tx=0x7efe6801c560 comp rx=0 tx=0).stop 2024-04-25T11:20:44.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe7a20a640 1 -- 172.21.15.106:0/2268402166 shutdown_connections 2024-04-25T11:20:44.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe7a20a640 1 --2- 172.21.15.106:0/2268402166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe740d1c80 0x7efe740b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe7a20a640 1 --2- 172.21.15.106:0/2268402166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe740d1740 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe7a20a640 1 -- 172.21.15.106:0/2268402166 >> 172.21.15.106:0/2268402166 conn(0x7efe7406c9f0 msgr2=0x7efe7406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe7a20a640 1 -- 172.21.15.106:0/2268402166 shutdown_connections 2024-04-25T11:20:44.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.047+0000 7efe7a20a640 1 -- 172.21.15.106:0/2268402166 wait complete. 2024-04-25T11:20:44.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 Processor -- start 2024-04-25T11:20:44.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 -- start start 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe74167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe740d1c80 0x7efe7415f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe74143fa0 0x7efe7415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efe7416ab10 con 0x7efe740cd930 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe7416a990 con 0x7efe740d1c80 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe7416ac90 con 0x7efe74143fa0 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe74167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe74167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60568/0 (socket says 172.21.15.106:60568) 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 -- 172.21.15.106:0/1848681866 learned_addr learned my addr 172.21.15.106:0/1848681866 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe73fff640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe74143fa0 0x7efe7415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe72ffd640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe740d1c80 0x7efe7415f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 -- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe74143fa0 msgr2=0x7efe7415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe74143fa0 0x7efe7415fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 -- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe740d1c80 msgr2=0x7efe7415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe740d1c80 0x7efe7415f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 -- 172.21.15.106:0/1848681866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe74160f70 con 0x7efe740cd930 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe72ffd640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe740d1c80 0x7efe7415f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe737fe640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe74167fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c3a0 tx=0x7efe6400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe5bfff640 1 -- 172.21.15.106:0/1848681866 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640052d0 con 0x7efe740cd930 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe5bfff640 1 -- 172.21.15.106:0/1848681866 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe64005470 con 0x7efe740cd930 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7415ddc0 con 0x7efe740cd930 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe5bfff640 1 -- 172.21.15.106:0/1848681866 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64007260 con 0x7efe740cd930 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7415d490 con 0x7efe740cd930 2024-04-25T11:20:44.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.051+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe74050750 con 0x7efe740cd930 2024-04-25T11:20:44.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.055+0000 7efe5bfff640 1 -- 172.21.15.106:0/1848681866 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe64007400 con 0x7efe740cd930 2024-04-25T11:20:44.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.055+0000 7efe5bfff640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe6006f400 0x7efe60071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.055+0000 7efe72ffd640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe6006f400 0x7efe60071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.055+0000 7efe5bfff640 1 -- 172.21.15.106:0/1848681866 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efe6408e770 con 0x7efe740cd930 2024-04-25T11:20:44.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.055+0000 7efe72ffd640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe6006f400 0x7efe60071910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe680af000 tx=0x7efe68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.059+0000 7efe5bfff640 1 -- 172.21.15.106:0/1848681866 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe740cdd40 con 0x7efe740cd930 2024-04-25T11:20:44.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.207+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe740b4780 con 0x7efe740cd930 2024-04-25T11:20:44.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.207+0000 7efe5bfff640 1 -- 172.21.15.106:0/1848681866 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7efe6405b660 con 0x7efe740cd930 2024-04-25T11:20:44.208 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:44.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe6006f400 msgr2=0x7efe60071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe6006f400 0x7efe60071910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe680af000 tx=0x7efe68008040 comp rx=0 tx=0).stop 2024-04-25T11:20:44.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 msgr2=0x7efe74167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe74167fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c3a0 tx=0x7efe6400c860 comp rx=0 tx=0).stop 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 shutdown_connections 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe74143fa0 0x7efe7415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe740d1c80 0x7efe7415f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7efe6006f400 0x7efe60071910 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 --2- 172.21.15.106:0/1848681866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe740cd930 0x7efe74167fb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 >> 172.21.15.106:0/1848681866 conn(0x7efe7406c9f0 msgr2=0x7efe74140b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 shutdown_connections 2024-04-25T11:20:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.211+0000 7efe7a20a640 1 -- 172.21.15.106:0/1848681866 wait complete. 2024-04-25T11:20:44.227 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:44.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfc1ce640 1 Processor -- start 2024-04-25T11:20:44.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfc1ce640 1 -- start start 2024-04-25T11:20:44.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfc1ce640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1df40b6120 0x7f1df415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfc1ce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40706c0 0x7f1df4072a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfc1ce640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df415c520 con 0x7f1df4072f60 2024-04-25T11:20:44.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfc1ce640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1df40b6530 con 0x7f1df40b6120 2024-04-25T11:20:44.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfc1ce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1df40b5000 con 0x7f1df40706c0 2024-04-25T11:20:44.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfa76b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1df4072f60 0x7f1df406ea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52666/0 (socket says 172.21.15.106:52666) 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfa76b640 1 -- 172.21.15.106:0/2065558709 learned_addr learned my addr 172.21.15.106:0/2065558709 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3224455359 0 0) 0x7f1df415c520 con 0x7f1df4072f60 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2065558709 >> v2:172.21.15.134:3300/0 conn(0x7f1df40b6120 0x7f1df415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9769640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40706c0 0x7f1df4072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1de0004640 con 0x7f1df4072f60 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1786388743 0 0) 0x7f1de0004640 con 0x7f1df4072f60 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40706c0 msgr2=0x7f1df4072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40706c0 0x7f1df4072a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 >> v2:172.21.15.134:3300/0 conn(0x7f1df40b6120 msgr2=0x7f1df415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 --2- 172.21.15.106:0/2065558709 >> v2:172.21.15.134:3300/0 conn(0x7f1df40b6120 0x7f1df415bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df4073370 con 0x7f1df4072f60 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f1dec001a00 con 0x7f1df4072f60 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2065558709 >> v2:172.21.15.134:3300/0 conn(0x7f1df40b6120 0x7f1df415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:44.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 >> v1:172.21.15.106:6789/0 conn(0x7f1df4072f60 legacy=0x7f1df406ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de0004ad0 0x7f1de0004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0005920 0x7f1de0005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40706c0 0x7f1de0006270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1df40b5000 con 0x7f1de0004ad0 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1de000cb90 con 0x7f1de0005920 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1de000cd10 con 0x7f1df40706c0 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de0004ad0 0x7f1de0004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9769640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0005920 0x7f1de0005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1dfa76b640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40706c0 0x7f1de0006270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9f6a640 1 -- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40706c0 msgr2=0x7f1de0006270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40706c0 0x7f1de0006270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9f6a640 1 -- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0005920 msgr2=0x7f1de0005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0005920 0x7f1de0005d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.367+0000 7f1df9f6a640 1 -- 172.21.15.106:0/2065558709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de0004640 con 0x7f1de0004ad0 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de0004ad0 0x7f1de0004f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009890 tx=0x7f1df001e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4bd1c39a446aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df000f040 con 0x7f1de0004ad0 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df00a86f0 con 0x7f1de0004ad0 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1df8f68640 1 -- 172.21.15.106:0/2065558709 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df00a8aa0 con 0x7f1de0004ad0 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2065558709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de0004ad0 msgr2=0x7f1de0004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1dfc1ce640 1 --2- 172.21.15.106:0/2065558709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de0004ad0 0x7f1de0004f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009890 tx=0x7f1df001e860 comp rx=0 tx=0).stop 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1dfa76b640 1 -- 172.21.15.106:0/2065558709 reap_dead start 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2065558709 shutdown_connections 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2065558709 >> 172.21.15.106:0/2065558709 conn(0x7f1df406c9f0 msgr2=0x7f1df406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2065558709 shutdown_connections 2024-04-25T11:20:44.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.371+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2065558709 wait complete. 2024-04-25T11:20:44.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 Processor -- start 2024-04-25T11:20:44.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 -- start start 2024-04-25T11:20:44.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0004ad0 0x7f1df415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df413ed10 0x7f1df4151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40b2030 0x7f1df4156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4070ad0 con 0x7f1df413ed10 2024-04-25T11:20:44.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1df40b4c90 con 0x7f1de0004ad0 2024-04-25T11:20:44.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1df4070c50 con 0x7f1df40b2030 2024-04-25T11:20:44.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df413ed10 0x7f1df4151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df413ed10 0x7f1df4151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60580/0 (socket says 172.21.15.106:60580) 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 -- 172.21.15.106:0/2235621281 learned_addr learned my addr 172.21.15.106:0/2235621281 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0004ad0 0x7f1df415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfa76b640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40b2030 0x7f1df4156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 -- 172.21.15.106:0/2235621281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40b2030 msgr2=0x7f1df4156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40b2030 0x7f1df4156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 -- 172.21.15.106:0/2235621281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0004ad0 msgr2=0x7f1df415caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0004ad0 0x7f1df415caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 -- 172.21.15.106:0/2235621281 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df415cfe0 con 0x7f1df413ed10 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1df9769640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df413ed10 0x7f1df4151dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1de4011610 tx=0x7f1de4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dea7fc640 1 -- 172.21.15.106:0/2235621281 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de400f8f0 con 0x7f1df413ed10 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dea7fc640 1 -- 172.21.15.106:0/2235621281 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de400fa90 con 0x7f1df413ed10 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dea7fc640 1 -- 172.21.15.106:0/2235621281 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de400fe40 con 0x7f1df413ed10 2024-04-25T11:20:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df40b1270 con 0x7f1df413ed10 2024-04-25T11:20:44.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.375+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df40b1830 con 0x7f1df413ed10 2024-04-25T11:20:44.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.379+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df4050750 con 0x7f1df413ed10 2024-04-25T11:20:44.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.379+0000 7f1dea7fc640 1 -- 172.21.15.106:0/2235621281 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1de4015460 con 0x7f1df413ed10 2024-04-25T11:20:44.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.379+0000 7f1dea7fc640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1dd806f430 0x7f1dd8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.379+0000 7f1dea7fc640 1 -- 172.21.15.106:0/2235621281 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1de408fb70 con 0x7f1df413ed10 2024-04-25T11:20:44.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.379+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1dd806f430 0x7f1dd8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.379+0000 7f1df9f6a640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1dd806f430 0x7f1dd8071940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1df00b1000 tx=0x7f1df00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.383+0000 7f1dea7fc640 1 -- 172.21.15.106:0/2235621281 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1de4019030 con 0x7f1df413ed10 2024-04-25T11:20:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.535+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1df406fec0 con 0x7f1df413ed10 2024-04-25T11:20:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.535+0000 7f1dea7fc640 1 -- 172.21.15.106:0/2235621281 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f1de405caf0 con 0x7f1df413ed10 2024-04-25T11:20:44.536 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1dd806f430 msgr2=0x7f1dd8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1dd806f430 0x7f1dd8071940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1df00b1000 tx=0x7f1df00046d0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df413ed10 msgr2=0x7f1df4151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df413ed10 0x7f1df4151dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1de4011610 tx=0x7f1de4011ad0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 shutdown_connections 2024-04-25T11:20:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df40b2030 0x7f1df4156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1dd806f430 0x7f1dd8071940 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df413ed10 0x7f1df4151dc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 --2- 172.21.15.106:0/2235621281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de0004ad0 0x7f1df415caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 >> 172.21.15.106:0/2235621281 conn(0x7f1df406c9f0 msgr2=0x7f1df41552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 shutdown_connections 2024-04-25T11:20:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.539+0000 7f1dfc1ce640 1 -- 172.21.15.106:0/2235621281 wait complete. 2024-04-25T11:20:44.556 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:44.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f74091640 1 Processor -- start 2024-04-25T11:20:44.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f74091640 1 -- start start 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f74091640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1f6c15dba0 0x7f1f6c1616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f74091640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f6c161be0 0x7f1f6c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f74091640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f74091640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6c0af390 con 0x7f1f6c161be0 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f74091640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6c0b6ec0 con 0x7f1f6c15dba0 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f74091640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c162530 con 0x7f1f6c0b59e0 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7162c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f6c161be0 0x7f1f6c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7162c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f6c161be0 0x7f1f6c161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60590/0 (socket says 172.21.15.106:60590) 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7162c640 1 -- 172.21.15.106:0/4076900058 learned_addr learned my addr 172.21.15.106:0/4076900058 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f71e2d640 1 --2- 172.21.15.106:0/4076900058 >> v2:172.21.15.134:3300/0 conn(0x7f1f6c15dba0 0x7f1f6c1616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7262e640 1 --2- 172.21.15.106:0/4076900058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7162c640 1 -- 172.21.15.106:0/4076900058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 msgr2=0x7f1f6c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7162c640 1 --2- 172.21.15.106:0/4076900058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c0b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7162c640 1 -- 172.21.15.106:0/4076900058 >> v2:172.21.15.134:3300/0 conn(0x7f1f6c15dba0 msgr2=0x7f1f6c1616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7162c640 1 --2- 172.21.15.106:0/4076900058 >> v2:172.21.15.134:3300/0 conn(0x7f1f6c15dba0 0x7f1f6c1616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7162c640 1 -- 172.21.15.106:0/4076900058 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c0aee10 con 0x7f1f6c161be0 2024-04-25T11:20:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f71e2d640 1 --2- 172.21.15.106:0/4076900058 >> v2:172.21.15.134:3300/0 conn(0x7f1f6c15dba0 0x7f1f6c1616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.703+0000 7f1f7262e640 1 --2- 172.21.15.106:0/4076900058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c0b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f7162c640 1 --2- 172.21.15.106:0/4076900058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c161ff0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c072a90 tx=0x7f1f6801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc5a9076ca27347 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f70e2b640 1 -- 172.21.15.106:0/4076900058 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680b0070 con 0x7f1f6c161be0 2024-04-25T11:20:44.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f70e2b640 1 -- 172.21.15.106:0/4076900058 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f680ab600 con 0x7f1f6c161be0 2024-04-25T11:20:44.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f70e2b640 1 -- 172.21.15.106:0/4076900058 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680ab9d0 con 0x7f1f6c161be0 2024-04-25T11:20:44.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 -- 172.21.15.106:0/4076900058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 msgr2=0x7f1f6c161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 --2- 172.21.15.106:0/4076900058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c161ff0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c072a90 tx=0x7f1f6801c520 comp rx=0 tx=0).stop 2024-04-25T11:20:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 -- 172.21.15.106:0/4076900058 shutdown_connections 2024-04-25T11:20:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 --2- 172.21.15.106:0/4076900058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c0b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 --2- 172.21.15.106:0/4076900058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c161ff0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 --2- 172.21.15.106:0/4076900058 >> v2:172.21.15.134:3300/0 conn(0x7f1f6c15dba0 0x7f1f6c1616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 -- 172.21.15.106:0/4076900058 >> 172.21.15.106:0/4076900058 conn(0x7f1f6c0d1ed0 msgr2=0x7f1f6c0d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 -- 172.21.15.106:0/4076900058 shutdown_connections 2024-04-25T11:20:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.707+0000 7f1f74091640 1 -- 172.21.15.106:0/4076900058 wait complete. 2024-04-25T11:20:44.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.711+0000 7f1f74091640 1 Processor -- start 2024-04-25T11:20:44.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.711+0000 7f1f74091640 1 -- start start 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.711+0000 7f1f74091640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c09f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.711+0000 7f1f74091640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c15dba0 0x7f1f6c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.711+0000 7f1f74091640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c0ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.711+0000 7f1f74091640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c06cfb0 con 0x7f1f6c161be0 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.711+0000 7f1f74091640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c0b6950 con 0x7f1f6c15dba0 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.711+0000 7f1f74091640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c06d130 con 0x7f1f6c0b59e0 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c0ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c0ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60592/0 (socket says 172.21.15.106:60592) 2024-04-25T11:20:44.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 -- 172.21.15.106:0/1797867032 learned_addr learned my addr 172.21.15.106:0/1797867032 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7162c640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c15dba0 0x7f1f6c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f71e2d640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c09f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 -- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 msgr2=0x7f1f6c09f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c09f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 -- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c15dba0 msgr2=0x7f1f6c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c15dba0 0x7f1f6c151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 -- 172.21.15.106:0/1797867032 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c0d07e0 con 0x7f1f6c161be0 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7162c640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c15dba0 0x7f1f6c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f7262e640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c0ceef0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c570 tx=0x7f1f5c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f627fc640 1 -- 172.21.15.106:0/1797867032 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c015020 con 0x7f1f6c161be0 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f627fc640 1 -- 172.21.15.106:0/1797867032 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c004440 con 0x7f1f6c161be0 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6c0ce710 con 0x7f1f6c161be0 2024-04-25T11:20:44.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f627fc640 1 -- 172.21.15.106:0/1797867032 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c003820 con 0x7f1f6c161be0 2024-04-25T11:20:44.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6c0d0d60 con 0x7f1f6c161be0 2024-04-25T11:20:44.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6c050750 con 0x7f1f6c161be0 2024-04-25T11:20:44.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f627fc640 1 -- 172.21.15.106:0/1797867032 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1f5c018020 con 0x7f1f6c161be0 2024-04-25T11:20:44.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f627fc640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1f5806f400 0x7f1f58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f627fc640 1 -- 172.21.15.106:0/1797867032 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1f5c08f200 con 0x7f1f6c161be0 2024-04-25T11:20:44.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f71e2d640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1f5806f400 0x7f1f58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.715+0000 7f1f71e2d640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1f5806f400 0x7f1f58071910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f640070b0 tx=0x7f1f64007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.719+0000 7f1f627fc640 1 -- 172.21.15.106:0/1797867032 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1f5c01f020 con 0x7f1f6c161be0 2024-04-25T11:20:44.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.891+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f6c152040 con 0x7f1f6c161be0 2024-04-25T11:20:44.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.891+0000 7f1f627fc640 1 -- 172.21.15.106:0/1797867032 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f1f5c05c180 con 0x7f1f6c161be0 2024-04-25T11:20:44.892 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:44.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1f5806f400 msgr2=0x7f1f58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1f5806f400 0x7f1f58071910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f640070b0 tx=0x7f1f64007040 comp rx=0 tx=0).stop 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 msgr2=0x7f1f6c0ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c0ceef0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c570 tx=0x7f1f5c00ca30 comp rx=0 tx=0).stop 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 shutdown_connections 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f1f5806f400 0x7f1f58071910 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f6c161be0 0x7f1f6c0ceef0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c15dba0 0x7f1f6c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 --2- 172.21.15.106:0/1797867032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f6c0b59e0 0x7f1f6c09f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 >> 172.21.15.106:0/1797867032 conn(0x7f1f6c0d1ed0 msgr2=0x7f1f6c15ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 shutdown_connections 2024-04-25T11:20:44.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:44.895+0000 7f1f74091640 1 -- 172.21.15.106:0/1797867032 wait complete. 2024-04-25T11:20:44.912 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-25T11:20:44.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:44.913 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:44.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[38150]: fuse finished with error 0 and tester_r 0 2024-04-25T11:20:44.928 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:51.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:20:51.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.029 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:20:51.037 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:20:51.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd007876640 1 Processor -- start 2024-04-25T11:20:51.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd007876640 1 -- start start 2024-04-25T11:20:51.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd007876640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00006cd00 0x7fd0000d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd007876640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd0000b1070 0x7fd0000b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd007876640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0000b3d30 con 0x7fd00015df60 2024-04-25T11:20:51.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd007876640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd0000d2290 con 0x7fd0000b1070 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd007876640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd0000d2030 con 0x7fd00006cd00 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd005e13640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd0000b1070 0x7fd0000b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd005e13640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd0000b1070 0x7fd0000b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48136/0 (socket says 172.21.15.106:48136) 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd005e13640 1 -- 172.21.15.106:0/41400183 learned_addr learned my addr 172.21.15.106:0/41400183 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd004e11640 1 --2- 172.21.15.106:0/41400183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00006cd00 0x7fd0000d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd005e13640 1 -- 172.21.15.106:0/41400183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00006cd00 msgr2=0x7fd0000d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd005e13640 1 --2- 172.21.15.106:0/41400183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00006cd00 0x7fd0000d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd005e13640 1 -- 172.21.15.106:0/41400183 >> v1:172.21.15.134:6789/0 conn(0x7fd00015df60 legacy=0x7fd00006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:51.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.239+0000 7fd005e13640 1 -- 172.21.15.106:0/41400183 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0000b4860 con 0x7fd0000b1070 2024-04-25T11:20:51.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd005e13640 1 --2- 172.21.15.106:0/41400183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd0000b1480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009e50 tx=0x7fcff801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742c1fb7a8fdd0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:51.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fcff7fff640 1 -- 172.21.15.106:0/41400183 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80a56c0 con 0x7fd0000b1070 2024-04-25T11:20:51.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fcff7fff640 1 -- 172.21.15.106:0/41400183 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff80a5860 con 0x7fd0000b1070 2024-04-25T11:20:51.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fcff7fff640 1 -- 172.21.15.106:0/41400183 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80a5c30 con 0x7fd0000b1070 2024-04-25T11:20:51.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd007876640 1 -- 172.21.15.106:0/41400183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 msgr2=0x7fd0000b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd007876640 1 --2- 172.21.15.106:0/41400183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd0000b1480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009e50 tx=0x7fcff801c910 comp rx=0 tx=0).stop 2024-04-25T11:20:51.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd007876640 1 -- 172.21.15.106:0/41400183 shutdown_connections 2024-04-25T11:20:51.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd007876640 1 --2- 172.21.15.106:0/41400183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd0000b1480 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd007876640 1 --2- 172.21.15.106:0/41400183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00006cd00 0x7fd0000d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd007876640 1 -- 172.21.15.106:0/41400183 >> 172.21.15.106:0/41400183 conn(0x7fd0000b7840 msgr2=0x7fd0000b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:51.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd007876640 1 -- 172.21.15.106:0/41400183 shutdown_connections 2024-04-25T11:20:51.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.243+0000 7fd007876640 1 -- 172.21.15.106:0/41400183 wait complete. 2024-04-25T11:20:51.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 Processor -- start 2024-04-25T11:20:51.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 -- start start 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd000140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00015df60 0x7fd000160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000160a70 0x7fd00015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd00016a8c0 con 0x7fd0000b1070 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd00016aa40 con 0x7fd000160a70 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd00016abc0 con 0x7fd00015df60 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd000140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd000140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48144/0 (socket says 172.21.15.106:48144) 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 -- 172.21.15.106:0/2321465061 learned_addr learned my addr 172.21.15.106:0/2321465061 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd004e11640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00015df60 0x7fd000160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005e13640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000160a70 0x7fd00015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 -- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00015df60 msgr2=0x7fd000160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00015df60 0x7fd000160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 -- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000160a70 msgr2=0x7fd00015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000160a70 0x7fd00015e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 -- 172.21.15.106:0/2321465061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd000161180 con 0x7fd0000b1070 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd004e11640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00015df60 0x7fd000160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd005612640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd000140dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c370 tx=0x7fcffc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fcff5ffb640 1 -- 172.21.15.106:0/2321465061 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0052d0 con 0x7fd0000b1070 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fcff5ffb640 1 -- 172.21.15.106:0/2321465061 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc005470 con 0x7fd0000b1070 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd000165ca0 con 0x7fd0000b1070 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fcff5ffb640 1 -- 172.21.15.106:0/2321465061 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc007260 con 0x7fd0000b1070 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.247+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd000166260 con 0x7fd0000b1070 2024-04-25T11:20:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.251+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd000157670 con 0x7fd0000b1070 2024-04-25T11:20:51.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.251+0000 7fcff5ffb640 1 -- 172.21.15.106:0/2321465061 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcffc007400 con 0x7fd0000b1070 2024-04-25T11:20:51.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.251+0000 7fcff5ffb640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcfe406f400 0x7fcfe4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.251+0000 7fd004e11640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcfe406f400 0x7fcfe4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.251+0000 7fcff5ffb640 1 -- 172.21.15.106:0/2321465061 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fcffc08ead0 con 0x7fd0000b1070 2024-04-25T11:20:51.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.251+0000 7fd004e11640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcfe406f400 0x7fcfe4071910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcff0006fd0 tx=0x7fcff0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:51.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.255+0000 7fcff5ffb640 1 -- 172.21.15.106:0/2321465061 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcffc08ee20 con 0x7fd0000b1070 2024-04-25T11:20:51.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.455+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd000166640 con 0x7fd0000b1070 2024-04-25T11:20:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.455+0000 7fcff5ffb640 1 -- 172.21.15.106:0/2321465061 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fcffc05bb30 con 0x7fd0000b1070 2024-04-25T11:20:51.455 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:20:51.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcfe406f400 msgr2=0x7fcfe4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcfe406f400 0x7fcfe4071910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcff0006fd0 tx=0x7fcff0008040 comp rx=0 tx=0).stop 2024-04-25T11:20:51.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 msgr2=0x7fd000140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd000140dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c370 tx=0x7fcffc00c830 comp rx=0 tx=0).stop 2024-04-25T11:20:51.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 shutdown_connections 2024-04-25T11:20:51.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000160a70 0x7fd00015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd00015df60 0x7fd000160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fcfe406f400 0x7fcfe4071910 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 --2- 172.21.15.106:0/2321465061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0000b1070 0x7fd000140dd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 >> 172.21.15.106:0/2321465061 conn(0x7fd0000b7840 msgr2=0x7fd000157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:51.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 shutdown_connections 2024-04-25T11:20:51.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51.459+0000 7fd007876640 1 -- 172.21.15.106:0/2321465061 wait complete. 2024-04-25T11:20:51.478 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:20:51.478 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:20:51.479 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-25T11:20:51.479 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:20:51.479 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:51.479 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:20:51.479 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:20:51.479 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:20:51.479 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:51.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.480 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-25T11:20:51.487 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:20:51.487 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:20:51.487 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:20:51.487 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.487 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-25T11:20:51.488 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.488 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:20:51.488 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.488 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-25T11:20:51.488 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:20:51.488 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.488 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-25T11:20:51.489 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T11:20:51.489 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.489 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:20:51.489 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.489 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:20:51.489 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42432sec preferred_lft 42432sec 2024-04-25T11:20:51.490 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-25T11:20:51.490 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.490 INFO:teuthology.orchestra.run.smithi106.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T11:20:51.490 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 02:42:4f:a1:63:df brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.490 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T11:20:51.490 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.490 INFO:teuthology.orchestra.run.smithi106.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:20:51.491 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.491 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:20:51.491 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.491 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T11:20:51.491 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.491 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:20:51.491 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:51.492 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T11:20:51.492 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.492 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:20:51.492 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:51.492 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T11:20:51.492 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.493 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:20:51.538 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-25T11:20:51.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.540 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-25T11:20:51.593 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T11:20:51.594 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:20:51.595 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:20:51.595 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T11:20:51.595 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:20:51.595 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:20:51.595 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:20:51.595 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T11:20:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:20:51.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:51.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.717 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T11:20:51.722 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:20:51.723 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:20:51.723 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:51.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.723 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-25T11:20:51.777 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T11:20:51.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:20:51.777 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:51.825 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:51.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:20:51.826 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:51.874 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:20:51.930 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:20:51.980 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:20:51.981 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:20:51.982 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:20:51.982 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:20:51.982 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:20:51.982 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:20:51.982 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:51.982 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:51.982 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:51.982 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:51.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.983 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:20:52.029 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T11:20:52.029 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T11:20:52.074 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:20:52.088 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:20:52.138 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:20:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:52.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:52.140 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:20:52.189 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:2024-04-25T11:20:52.191+0000 7fc743b6b580 -1 init, newargv = 0x7fc72801b4a0 newargc=15 2024-04-25T11:20:52.189 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[38530]: starting ceph client 2024-04-25T11:20:52.194 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[38530]: starting fuse 2024-04-25T11:20:53.286 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T11:20:53.302 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:20:53.350 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/run/lock 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:20:53.351 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/fs/nfsd 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/run/rpc_pipefs 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:20:53.352 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T11:20:53.353 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:53.353 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:53.353 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:53.353 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:53.353 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:53.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.354 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T11:20:53.405 INFO:teuthology.orchestra.run.smithi106.stdout:45 2024-04-25T11:20:53.406 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T11:20:53.406 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:20:53.406 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-25T11:20:53.406 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-25T11:20:53.406 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-25T11:20:53.406 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-25T11:20:53.406 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:53.406 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-25T11:20:53.406 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:20:53.407 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-25T11:20:53.408 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-25T11:20:53.408 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-25T11:20:53.408 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:20:53.408 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:53.408 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-25T11:20:53.408 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-25T11:20:53.530 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.38530.asok 2024-04-25T11:20:53.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:53.542 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38530.asok 2024-04-25T11:20:53.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.543 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38530.asok status 2024-04-25T11:20:53.718 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:20:53.718 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "0", 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "38530", 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5484, 2024-04-25T11:20:53.719 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 5484 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 1215212184 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T11:20:53.720 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 1215212184 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.5484 192.168.0.1:0/1215212184", 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/1215212184", 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 17, 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 33, 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-25T11:20:53.721 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:20:53.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.736 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-25T11:20:53.736 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:53.743 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-25T11:20:53.743 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:53.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.743 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-25T11:20:53.798 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:53.799 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-25T11:20:53.800 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-25T11:20:53.918 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.38530.asok 2024-04-25T11:20:53.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:53.930 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38530.asok 2024-04-25T11:20:53.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.931 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38530.asok perf dump objecter 2024-04-25T11:20:54.108 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 0, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 0, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 0, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 0, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:54.109 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 0, 2024-04-25T11:20:54.110 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-04-25T11:20:54.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-04-25T11:20:54.112 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 1, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 0, 2024-04-25T11:20:54.113 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-04-25T11:20:54.114 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-04-25T11:20:54.114 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 0, 2024-04-25T11:20:54.114 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 0, 2024-04-25T11:20:54.114 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 0 2024-04-25T11:20:54.114 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:54.114 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:20:54.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:54.125 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=foo of=/dev/null bs=128k count=32) 2024-04-25T11:20:54.159 INFO:teuthology.orchestra.run.smithi106.stderr:32+0 records in 2024-04-25T11:20:54.159 INFO:teuthology.orchestra.run.smithi106.stderr:32+0 records out 2024-04-25T11:20:54.159 INFO:teuthology.orchestra.run.smithi106.stderr:4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.027193 s, 154 MB/s 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-25T11:20:54.160 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:20:54.161 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T11:20:54.162 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-25T11:20:54.162 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-25T11:20:54.283 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.38530.asok 2024-04-25T11:20:54.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:54.294 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38530.asok 2024-04-25T11:20:54.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:54.295 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38530.asok perf dump objecter 2024-04-25T11:20:54.476 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:20:54.476 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-04-25T11:20:54.476 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-04-25T11:20:54.476 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-04-25T11:20:54.476 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 10, 2024-04-25T11:20:54.476 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 0, 2024-04-25T11:20:54.476 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 0, 2024-04-25T11:20:54.476 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 10, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.047999383, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.004799938 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 10 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 10, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 8, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 2, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-04-25T11:20:54.477 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 1, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 1, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 8, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-04-25T11:20:54.478 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-04-25T11:20:54.479 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 1, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 0, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 2, 2024-04-25T11:20:54.480 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 2, 2024-04-25T11:20:54.481 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-04-25T11:20:54.481 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-04-25T11:20:54.481 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 0, 2024-04-25T11:20:54.481 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 0, 2024-04-25T11:20:54.481 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 0 2024-04-25T11:20:54.481 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T11:20:54.481 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:20:54.493 INFO:tasks.cephfs.test_readahead:read operations: 8 2024-04-25T11:20:54.493 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:20:54.494 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:20:54.543 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:20:54.589 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:20:54.589 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:20:54.589 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:54.589 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:54.595 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:54.595 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:54.641 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f26960a3640 1 Processor -- start 2024-04-25T11:20:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f26960a3640 1 -- start start 2024-04-25T11:20:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f26960a3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 0x7f26900d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f26960a3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f269006c5f0 0x7f269006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f26960a3640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269006cf40 con 0x7f26900b26a0 2024-04-25T11:20:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f26960a3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f26900d2290 con 0x7f269006c5f0 2024-04-25T11:20:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f26960a3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26900d2030 con 0x7f269015df60 2024-04-25T11:20:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f268effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f269006c5f0 0x7f269006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f268ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f26900b26a0 0x7f26900b2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47340/0 (socket says 172.21.15.106:47340) 2024-04-25T11:20:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f268ffff640 1 -- 172.21.15.106:0/4230666946 learned_addr learned my addr 172.21.15.106:0/4230666946 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:54.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f268e7fc640 1 -- 172.21.15.106:0/4230666946 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388868416 0 0) 0x7f26880014a0 con 0x7f26900b26a0 2024-04-25T11:20:54.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f268e7fc640 1 -- 172.21.15.106:0/4230666946 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2678003530 con 0x7f26900b26a0 2024-04-25T11:20:54.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f268f7fe640 1 --2- 172.21.15.106:0/4230666946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 0x7f26900d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.847+0000 7f268effd640 1 -- 172.21.15.106:0/4230666946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 msgr2=0x7f26900d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:54.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f268effd640 1 --2- 172.21.15.106:0/4230666946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 0x7f26900d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f268effd640 1 -- 172.21.15.106:0/4230666946 >> v1:172.21.15.106:6789/0 conn(0x7f26900b26a0 legacy=0x7f26900b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f268effd640 1 -- 172.21.15.106:0/4230666946 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26900b3d30 con 0x7f269006c5f0 2024-04-25T11:20:54.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f268effd640 1 --2- 172.21.15.106:0/4230666946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269006c5f0 0x7f269006ca00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f26840097c0 tx=0x7f268401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91aa1b9ca74f71b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:54.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f268e7fc640 1 -- 172.21.15.106:0/4230666946 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840b0070 con 0x7f269006c5f0 2024-04-25T11:20:54.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f268e7fc640 1 -- 172.21.15.106:0/4230666946 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26840ab600 con 0x7f269006c5f0 2024-04-25T11:20:54.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f268e7fc640 1 -- 172.21.15.106:0/4230666946 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840ab9d0 con 0x7f269006c5f0 2024-04-25T11:20:54.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f26960a3640 1 -- 172.21.15.106:0/4230666946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269006c5f0 msgr2=0x7f269006ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f26960a3640 1 --2- 172.21.15.106:0/4230666946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269006c5f0 0x7f269006ca00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f26840097c0 tx=0x7f268401c350 comp rx=0 tx=0).stop 2024-04-25T11:20:54.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f26960a3640 1 -- 172.21.15.106:0/4230666946 shutdown_connections 2024-04-25T11:20:54.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f26960a3640 1 --2- 172.21.15.106:0/4230666946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269006c5f0 0x7f269006ca00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f26960a3640 1 --2- 172.21.15.106:0/4230666946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 0x7f26900d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f26960a3640 1 -- 172.21.15.106:0/4230666946 >> 172.21.15.106:0/4230666946 conn(0x7f26900b7840 msgr2=0x7f26900b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:54.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f26960a3640 1 -- 172.21.15.106:0/4230666946 shutdown_connections 2024-04-25T11:20:54.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.851+0000 7f26960a3640 1 -- 172.21.15.106:0/4230666946 wait complete. 2024-04-25T11:20:54.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 Processor -- start 2024-04-25T11:20:54.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 -- start start 2024-04-25T11:20:54.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269006c5f0 0x7f269014cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26900b26a0 0x7f26901507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 0x7f2690150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f26900b2080 con 0x7f26900b26a0 2024-04-25T11:20:54.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26900b4e20 con 0x7f269006c5f0 2024-04-25T11:20:54.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26900b2200 con 0x7f269015df60 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26900b26a0 0x7f26901507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26900b26a0 0x7f26901507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48160/0 (socket says 172.21.15.106:48160) 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 -- 172.21.15.106:0/3742037692 learned_addr learned my addr 172.21.15.106:0/3742037692 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268f7fe640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269006c5f0 0x7f269014cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268ffff640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 0x7f2690150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 -- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 msgr2=0x7f2690150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 0x7f2690150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 -- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269006c5f0 msgr2=0x7f269014cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269006c5f0 0x7f269014cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 -- 172.21.15.106:0/3742037692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269015f4b0 con 0x7f26900b26a0 2024-04-25T11:20:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268f7fe640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269006c5f0 0x7f269014cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f268effd640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26900b26a0 0x7f26901507a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f268401aa00 tx=0x7f26840190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f2677fff640 1 -- 172.21.15.106:0/3742037692 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840b8020 con 0x7f26900b26a0 2024-04-25T11:20:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f2677fff640 1 -- 172.21.15.106:0/3742037692 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26840b4610 con 0x7f26900b26a0 2024-04-25T11:20:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26900d0880 con 0x7f26900b26a0 2024-04-25T11:20:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f2677fff640 1 -- 172.21.15.106:0/3742037692 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840b0040 con 0x7f26900b26a0 2024-04-25T11:20:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.855+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26900d0e40 con 0x7f26900b26a0 2024-04-25T11:20:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.859+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2690070560 con 0x7f26900b26a0 2024-04-25T11:20:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.859+0000 7f2677fff640 1 -- 172.21.15.106:0/3742037692 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f26840bb020 con 0x7f26900b26a0 2024-04-25T11:20:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.859+0000 7f2677fff640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f267c06f400 0x7f267c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.859+0000 7f268f7fe640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f267c06f400 0x7f267c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.859+0000 7f2677fff640 1 -- 172.21.15.106:0/3742037692 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2684129010 con 0x7f26900b26a0 2024-04-25T11:20:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.859+0000 7f268f7fe640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f267c06f400 0x7f267c071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2680006fd0 tx=0x7f2680008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:54.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:54.863+0000 7f2677fff640 1 -- 172.21.15.106:0/3742037692 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26840c2030 con 0x7f26900b26a0 2024-04-25T11:20:55.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.027+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f269015d280 con 0x7f26900b26a0 2024-04-25T11:20:55.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.027+0000 7f2677fff640 1 -- 172.21.15.106:0/3742037692 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7f2684016090 con 0x7f26900b26a0 2024-04-25T11:20:55.028 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:55.028 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:33.406557+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f267c06f400 msgr2=0x7f267c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f267c06f400 0x7f267c071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2680006fd0 tx=0x7f2680008040 comp rx=0 tx=0).stop 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26900b26a0 msgr2=0x7f26901507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26900b26a0 0x7f26901507a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f268401aa00 tx=0x7f26840190f0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 shutdown_connections 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269015df60 0x7f2690150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f267c06f400 0x7f267c071910 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26900b26a0 0x7f26901507a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 --2- 172.21.15.106:0/3742037692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269006c5f0 0x7f269014cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 >> 172.21.15.106:0/3742037692 conn(0x7f26900b7840 msgr2=0x7f269006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 shutdown_connections 2024-04-25T11:20:55.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.031+0000 7f26960a3640 1 -- 172.21.15.106:0/3742037692 wait complete. 2024-04-25T11:20:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:20:29.516176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:20:30.780121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:55.049 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:55.129 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:20:55.132 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:20:55.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face36fa640 1 Processor -- start 2024-04-25T11:20:55.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face36fa640 1 -- start start 2024-04-25T11:20:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face36fa640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7facdc0b4bd0 0x7facdc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face36fa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc0b36e0 con 0x7facdc0b25c0 2024-04-25T11:20:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face36fa640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc0b39c0 con 0x7facdc06c5f0 2024-04-25T11:20:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face36fa640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7facdc0b4fe0 con 0x7facdc0b4bd0 2024-04-25T11:20:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face1c97640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7facdc0b25c0 0x7facdc0b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47350/0 (socket says 172.21.15.106:47350) 2024-04-25T11:20:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face1c97640 1 -- 172.21.15.106:0/2522425967 learned_addr learned my addr 172.21.15.106:0/2522425967 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face1496640 1 --2- 172.21.15.106:0/2522425967 >> v2:172.21.15.134:3300/0 conn(0x7facdc0b4bd0 0x7facdc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1368395175 0 0) 0x7facdc0b36e0 con 0x7facdc0b25c0 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facc8003530 con 0x7facdc0b25c0 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 709315034 0 0) 0x7facc8003530 con 0x7facdc0b25c0 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 >> v2:172.21.15.134:3300/0 conn(0x7facdc0b4bd0 msgr2=0x7facdc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 --2- 172.21.15.106:0/2522425967 >> v2:172.21.15.134:3300/0 conn(0x7facdc0b4bd0 0x7facdc0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 >> v1:172.21.15.134:6789/0 conn(0x7facdc06c5f0 legacy=0x7facdc06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0b3ca0 con 0x7facdc0b25c0 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7facd4001a00 con 0x7facdc0b25c0 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 >> v1:172.21.15.106:6789/0 conn(0x7facdc0b25c0 legacy=0x7facdc0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc80039c0 0x7facc8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8004810 0x7facc8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80051d0 0x7facc800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0b4fe0 con 0x7facc80039c0 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7facc800c170 con 0x7facc8004810 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7facc800c370 con 0x7facc80051d0 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.199+0000 7face1496640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc80039c0 0x7facc8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face0c95640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8004810 0x7facc8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face1c97640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80051d0 0x7facc800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face1496640 1 -- 172.21.15.106:0/2522425967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80051d0 msgr2=0x7facc800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face1496640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80051d0 0x7facc800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face1496640 1 -- 172.21.15.106:0/2522425967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8004810 msgr2=0x7facc8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face1496640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8004810 0x7facc8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face1496640 1 -- 172.21.15.106:0/2522425967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc8003530 con 0x7facc80039c0 2024-04-25T11:20:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face1c97640 1 -- 172.21.15.106:0/2522425967 reap_dead start 2024-04-25T11:20:55.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face1496640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc80039c0 0x7facc8003e30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7facd8009ae0 tx=0x7facd801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe4586c694486e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80096b0 con 0x7facc80039c0 2024-04-25T11:20:55.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd80a9480 con 0x7facc80039c0 2024-04-25T11:20:55.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7facd3fff640 1 -- 172.21.15.106:0/2522425967 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80a9870 con 0x7facc80039c0 2024-04-25T11:20:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face36fa640 1 -- 172.21.15.106:0/2522425967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc80039c0 msgr2=0x7facc8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face36fa640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc80039c0 0x7facc8003e30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7facd8009ae0 tx=0x7facd801c9d0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face36fa640 1 -- 172.21.15.106:0/2522425967 shutdown_connections 2024-04-25T11:20:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face36fa640 1 --2- 172.21.15.106:0/2522425967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facc80039c0 0x7facc8003e30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face36fa640 1 -- 172.21.15.106:0/2522425967 >> 172.21.15.106:0/2522425967 conn(0x7facdc0b40b0 msgr2=0x7facdc0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face36fa640 1 -- 172.21.15.106:0/2522425967 shutdown_connections 2024-04-25T11:20:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.203+0000 7face36fa640 1 -- 172.21.15.106:0/2522425967 wait complete. 2024-04-25T11:20:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 Processor -- start 2024-04-25T11:20:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 -- start start 2024-04-25T11:20:55.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80039c0 0x7facdc158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facdc14b8e0 0x7facdc14bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facdc0b71b0 0x7facdc0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0b1ce0 con 0x7facdc0b71b0 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0b1b60 con 0x7facdc14b8e0 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0b2de0 con 0x7facc80039c0 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facdc0b71b0 0x7facdc0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facdc0b71b0 0x7facdc0b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48180/0 (socket says 172.21.15.106:48180) 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 -- 172.21.15.106:0/972551994 learned_addr learned my addr 172.21.15.106:0/972551994 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face0c95640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facdc14b8e0 0x7facdc14bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1496640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80039c0 0x7facdc158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 -- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80039c0 msgr2=0x7facdc158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80039c0 0x7facdc158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 -- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facdc14b8e0 msgr2=0x7facdc14bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facdc14b8e0 0x7facdc14bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 -- 172.21.15.106:0/972551994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc14c230 con 0x7facdc0b71b0 2024-04-25T11:20:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face0c95640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facdc14b8e0 0x7facdc14bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:20:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face1c97640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facdc0b71b0 0x7facdc0b75c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7facdc06fa40 tx=0x7facd400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7facd1ffb640 1 -- 172.21.15.106:0/972551994 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd400ec10 con 0x7facdc0b71b0 2024-04-25T11:20:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7facd1ffb640 1 -- 172.21.15.106:0/972551994 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4018070 con 0x7facdc0b71b0 2024-04-25T11:20:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7facd1ffb640 1 -- 172.21.15.106:0/972551994 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4013420 con 0x7facdc0b71b0 2024-04-25T11:20:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc0b7b00 con 0x7facdc0b71b0 2024-04-25T11:20:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.207+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc159080 con 0x7facdc0b71b0 2024-04-25T11:20:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.211+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc050750 con 0x7facdc0b71b0 2024-04-25T11:20:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.211+0000 7facd1ffb640 1 -- 172.21.15.106:0/972551994 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7facd40135c0 con 0x7facdc0b71b0 2024-04-25T11:20:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.211+0000 7facd1ffb640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facc006f400 0x7facc0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.211+0000 7face1496640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facc006f400 0x7facc0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.211+0000 7facd1ffb640 1 -- 172.21.15.106:0/972551994 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7facd408f960 con 0x7facdc0b71b0 2024-04-25T11:20:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.211+0000 7face1496640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facc006f400 0x7facc0071910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7facd80ae000 tx=0x7facd80046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.215+0000 7facd1ffb640 1 -- 172.21.15.106:0/972551994 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facdc0b7b00 con 0x7facdc0b71b0 2024-04-25T11:20:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.363+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7facdc158b30 con 0x7facdc0b71b0 2024-04-25T11:20:55.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.363+0000 7facd1ffb640 1 -- 172.21.15.106:0/972551994 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7facdc14c230 con 0x7facdc0b71b0 2024-04-25T11:20:55.364 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:55.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facc006f400 msgr2=0x7facc0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facc006f400 0x7facc0071910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7facd80ae000 tx=0x7facd80046d0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facdc0b71b0 msgr2=0x7facdc0b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facdc0b71b0 0x7facdc0b75c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7facdc06fa40 tx=0x7facd400e530 comp rx=0 tx=0).stop 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 shutdown_connections 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7facc006f400 0x7facc0071910 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facdc0b71b0 0x7facdc0b75c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facdc14b8e0 0x7facdc14bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 --2- 172.21.15.106:0/972551994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facc80039c0 0x7facdc158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 >> 172.21.15.106:0/972551994 conn(0x7facdc0b40b0 msgr2=0x7facdc15f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 shutdown_connections 2024-04-25T11:20:55.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.367+0000 7face36fa640 1 -- 172.21.15.106:0/972551994 wait complete. 2024-04-25T11:20:55.384 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:55.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f131cf80640 1 Processor -- start 2024-04-25T11:20:55.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f131cf80640 1 -- start start 2024-04-25T11:20:55.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f131cf80640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d0cf0 0x7f131806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f131cf80640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f131815d230 0x7f131815d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f131cf80640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13180b68e0 con 0x7f13180d1740 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f131cf80640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1318161210 con 0x7f131815d230 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f131cf80640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13180b6290 con 0x7f13180d0cf0 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f1316d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f131815d230 0x7f131815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f1316d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f131815d230 0x7f131815d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49532/0 (socket says 172.21.15.106:49532) 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f1316d9d640 1 -- 172.21.15.106:0/1332141923 learned_addr learned my addr 172.21.15.106:0/1332141923 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f1316d9d640 1 -- 172.21.15.106:0/1332141923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d0cf0 msgr2=0x7f131806ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f1316d9d640 1 --2- 172.21.15.106:0/1332141923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d0cf0 0x7f131806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f1316d9d640 1 -- 172.21.15.106:0/1332141923 >> v1:172.21.15.134:6789/0 conn(0x7f13180d1740 legacy=0x7f13180d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.527+0000 7f1316d9d640 1 -- 172.21.15.106:0/1332141923 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13180b6570 con 0x7f131815d230 2024-04-25T11:20:55.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f1316d9d640 1 --2- 172.21.15.106:0/1332141923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f131815d640 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f13180729b0 tx=0x7f131001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0705dd180811bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131559a640 1 -- 172.21.15.106:0/1332141923 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100a5550 con 0x7f131815d230 2024-04-25T11:20:55.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131559a640 1 -- 172.21.15.106:0/1332141923 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13100a56f0 con 0x7f131815d230 2024-04-25T11:20:55.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131559a640 1 -- 172.21.15.106:0/1332141923 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100a5ac0 con 0x7f131815d230 2024-04-25T11:20:55.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131cf80640 1 -- 172.21.15.106:0/1332141923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 msgr2=0x7f131815d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131cf80640 1 --2- 172.21.15.106:0/1332141923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f131815d640 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f13180729b0 tx=0x7f131001c8d0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131cf80640 1 -- 172.21.15.106:0/1332141923 shutdown_connections 2024-04-25T11:20:55.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131cf80640 1 --2- 172.21.15.106:0/1332141923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f131815d640 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131cf80640 1 --2- 172.21.15.106:0/1332141923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d0cf0 0x7f131806ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131cf80640 1 -- 172.21.15.106:0/1332141923 >> 172.21.15.106:0/1332141923 conn(0x7f131806c9f0 msgr2=0x7f131806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131cf80640 1 -- 172.21.15.106:0/1332141923 shutdown_connections 2024-04-25T11:20:55.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.531+0000 7f131cf80640 1 -- 172.21.15.106:0/1332141923 wait complete. 2024-04-25T11:20:55.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 Processor -- start 2024-04-25T11:20:55.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 -- start start 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d0cf0 0x7f13181427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1740 0x7f1318141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f1318158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f131816a080 con 0x7f131815d230 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f131815cdd0 con 0x7f13180d1740 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f131816a200 con 0x7f13180d0cf0 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f1318158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f1318158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49546/0 (socket says 172.21.15.106:49546) 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 -- 172.21.15.106:0/2919400155 learned_addr learned my addr 172.21.15.106:0/2919400155 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 -- 172.21.15.106:0/2919400155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d0cf0 msgr2=0x7f13181427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d0cf0 0x7f13181427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 -- 172.21.15.106:0/2919400155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1740 msgr2=0x7f1318141c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1740 0x7f1318141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 -- 172.21.15.106:0/2919400155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1318157460 con 0x7f131815d230 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1316d9d640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f1318158500 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f13100af040 tx=0x7f1310019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1306ffd640 1 -- 172.21.15.106:0/2919400155 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100bc020 con 0x7f131815d230 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1306ffd640 1 -- 172.21.15.106:0/2919400155 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13100b8690 con 0x7f131815d230 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1306ffd640 1 -- 172.21.15.106:0/2919400155 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100b8a80 con 0x7f131815d230 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1318157740 con 0x7f131815d230 2024-04-25T11:20:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1318157d00 con 0x7f131815d230 2024-04-25T11:20:55.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1318050750 con 0x7f131815d230 2024-04-25T11:20:55.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.535+0000 7f1306ffd640 1 -- 172.21.15.106:0/2919400155 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13100bf020 con 0x7f131815d230 2024-04-25T11:20:55.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.539+0000 7f1306ffd640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12f806f400 0x7f12f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.539+0000 7f1306ffd640 1 -- 172.21.15.106:0/2919400155 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f131012a6a0 con 0x7f131815d230 2024-04-25T11:20:55.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.539+0000 7f131659c640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12f806f400 0x7f12f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.539+0000 7f131659c640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12f806f400 0x7f12f8071910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f13180721f0 tx=0x7f1308006c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.543+0000 7f1306ffd640 1 -- 172.21.15.106:0/2919400155 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13100c6030 con 0x7f131815d230 2024-04-25T11:20:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.687+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f13180b1380 con 0x7f131815d230 2024-04-25T11:20:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f1306ffd640 1 -- 172.21.15.106:0/2919400155 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f13100165e0 con 0x7f131815d230 2024-04-25T11:20:55.689 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12f806f400 msgr2=0x7f12f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12f806f400 0x7f12f8071910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f13180721f0 tx=0x7f1308006c00 comp rx=0 tx=0).stop 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 msgr2=0x7f1318158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f1318158500 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f13100af040 tx=0x7f1310019670 comp rx=0 tx=0).stop 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 shutdown_connections 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12f806f400 0x7f12f8071910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131815d230 0x7f1318158500 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1740 0x7f1318141c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 --2- 172.21.15.106:0/2919400155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d0cf0 0x7f13181427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 >> 172.21.15.106:0/2919400155 conn(0x7f131806c9f0 msgr2=0x7f1318166dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 shutdown_connections 2024-04-25T11:20:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.691+0000 7f131cf80640 1 -- 172.21.15.106:0/2919400155 wait complete. 2024-04-25T11:20:55.708 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:55.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc82fd1640 1 Processor -- start 2024-04-25T11:20:55.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc82fd1640 1 -- start start 2024-04-25T11:20:55.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc82fd1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdc7c0ceb70 0x7fdc7c0cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc82fd1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c0d2850 con 0x7fdc7c0d1f20 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc82fd1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c0cf7c0 con 0x7fdc7c0d1b10 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc82fd1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdc7c0d0f90 con 0x7fdc7c0ceb70 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc7bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdc7c0d1f20 0x7fdc7c0ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44906/0 (socket says 172.21.15.106:44906) 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc7bfff640 1 -- 172.21.15.106:0/1159536761 learned_addr learned my addr 172.21.15.106:0/1159536761 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc8156e640 1 --2- 172.21.15.106:0/1159536761 >> v2:172.21.15.134:3300/0 conn(0x7fdc7c0ceb70 0x7fdc7c0cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 435652645 0 0) 0x7fdc7c0d2850 con 0x7fdc7c0d1f20 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.863+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc64003530 con 0x7fdc7c0d1f20 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 810904301 0 0) 0x7fdc64003530 con 0x7fdc7c0d1f20 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 >> v2:172.21.15.134:3300/0 conn(0x7fdc7c0ceb70 msgr2=0x7fdc7c0cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 --2- 172.21.15.106:0/1159536761 >> v2:172.21.15.134:3300/0 conn(0x7fdc7c0ceb70 0x7fdc7c0cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 >> v1:172.21.15.134:6789/0 conn(0x7fdc7c0d1b10 legacy=0x7fdc7c06dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c0b8b60 con 0x7fdc7c0d1f20 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fdc70001680 con 0x7fdc7c0d1f20 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 >> v1:172.21.15.106:6789/0 conn(0x7fdc7c0d1f20 legacy=0x7fdc7c0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640039c0 0x7fdc64003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64004810 0x7fdc64004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc64005260 0x7fdc6400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6400c1c0 con 0x7fdc64004810 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6400c370 con 0x7fdc640039c0 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6400c570 con 0x7fdc64005260 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7bfff640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64004810 0x7fdc64004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7bfff640 1 -- 172.21.15.106:0/1159536761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc64005260 msgr2=0x7fdc6400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7bfff640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc64005260 0x7fdc6400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7bfff640 1 -- 172.21.15.106:0/1159536761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640039c0 msgr2=0x7fdc64003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc80d6d640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640039c0 0x7fdc64003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7bfff640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640039c0 0x7fdc64003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc8156e640 1 -- 172.21.15.106:0/1159536761 reap_dead start 2024-04-25T11:20:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7bfff640 1 -- 172.21.15.106:0/1159536761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc64003530 con 0x7fdc64004810 2024-04-25T11:20:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc80d6d640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640039c0 0x7fdc64003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7bfff640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64004810 0x7fdc64004c20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0cdb70 tx=0x7fdc7001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d22c45c56a0e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70007d40 con 0x7fdc64004810 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc700a9070 con 0x7fdc64004810 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc7b7fe640 1 -- 172.21.15.106:0/1159536761 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70005d80 con 0x7fdc64004810 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc82fd1640 1 -- 172.21.15.106:0/1159536761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64004810 msgr2=0x7fdc64004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc82fd1640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64004810 0x7fdc64004c20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0cdb70 tx=0x7fdc7001cc20 comp rx=0 tx=0).stop 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc82fd1640 1 -- 172.21.15.106:0/1159536761 shutdown_connections 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc82fd1640 1 --2- 172.21.15.106:0/1159536761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64004810 0x7fdc64004c20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc82fd1640 1 -- 172.21.15.106:0/1159536761 >> 172.21.15.106:0/1159536761 conn(0x7fdc7c0cfff0 msgr2=0x7fdc7c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc82fd1640 1 -- 172.21.15.106:0/1159536761 shutdown_connections 2024-04-25T11:20:55.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.867+0000 7fdc82fd1640 1 -- 172.21.15.106:0/1159536761 wait complete. 2024-04-25T11:20:55.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc82fd1640 1 Processor -- start 2024-04-25T11:20:55.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc82fd1640 1 -- start start 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc82fd1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc64004810 0x7fdc7c1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc82fd1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc7c162dc0 0x7fdc7c1f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc82fd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc7c09f9b0 0x7fdc7c1ecb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc82fd1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c0ce830 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc82fd1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c06e790 con 0x7fdc64004810 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc82fd1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c0ce9b0 con 0x7fdc7c162dc0 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc7c09f9b0 0x7fdc7c1ecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc7c09f9b0 0x7fdc7c1ecb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49556/0 (socket says 172.21.15.106:49556) 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 -- 172.21.15.106:0/2821612551 learned_addr learned my addr 172.21.15.106:0/2821612551 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc7bfff640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc7c162dc0 0x7fdc7c1f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc80d6d640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc64004810 0x7fdc7c1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 -- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc7c162dc0 msgr2=0x7fdc7c1f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc7c162dc0 0x7fdc7c1f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 -- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc64004810 msgr2=0x7fdc7c1620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc64004810 0x7fdc7c1620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 -- 172.21.15.106:0/2821612551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c0b3690 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc7bfff640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc7c162dc0 0x7fdc7c1f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.871+0000 7fdc8156e640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc7c09f9b0 0x7fdc7c1ecb60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c073dd0 tx=0x7fdc7400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc797fa640 1 -- 172.21.15.106:0/2821612551 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7400bd10 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc797fa640 1 -- 172.21.15.106:0/2821612551 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc7401a070 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c0727e0 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc797fa640 1 -- 172.21.15.106:0/2821612551 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74014640 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c09eed0 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c050750 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc797fa640 1 -- 172.21.15.106:0/2821612551 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdc74013070 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc797fa640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdc6806f400 0x7fdc68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc797fa640 1 -- 172.21.15.106:0/2821612551 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdc7408fef0 con 0x7fdc7c09f9b0 2024-04-25T11:20:55.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc80d6d640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdc6806f400 0x7fdc68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.875+0000 7fdc80d6d640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdc6806f400 0x7fdc68071910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c000c10 tx=0x7fdc6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:55.879+0000 7fdc797fa640 1 -- 172.21.15.106:0/2821612551 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdc7c0b3690 con 0x7fdc7c09f9b0 2024-04-25T11:20:56.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.039+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc7c0b6c50 con 0x7fdc7c09f9b0 2024-04-25T11:20:56.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.039+0000 7fdc797fa640 1 -- 172.21.15.106:0/2821612551 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fdc7405cec0 con 0x7fdc7c09f9b0 2024-04-25T11:20:56.040 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:56.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdc6806f400 msgr2=0x7fdc68071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdc6806f400 0x7fdc68071910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c000c10 tx=0x7fdc6c009040 comp rx=0 tx=0).stop 2024-04-25T11:20:56.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc7c09f9b0 msgr2=0x7fdc7c1ecb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc7c09f9b0 0x7fdc7c1ecb60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c073dd0 tx=0x7fdc7400b480 comp rx=0 tx=0).stop 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 shutdown_connections 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdc6806f400 0x7fdc68071910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc7c09f9b0 0x7fdc7c1ecb60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc7c162dc0 0x7fdc7c1f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 --2- 172.21.15.106:0/2821612551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc64004810 0x7fdc7c1620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 >> 172.21.15.106:0/2821612551 conn(0x7fdc7c0cfff0 msgr2=0x7fdc7c164d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 shutdown_connections 2024-04-25T11:20:56.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.043+0000 7fdc82fd1640 1 -- 172.21.15.106:0/2821612551 wait complete. 2024-04-25T11:20:56.061 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25ae83d640 1 Processor -- start 2024-04-25T11:20:56.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25ae83d640 1 -- start start 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25ae83d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25a8073460 0x7f25a815e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25ae83d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f25a806cf30 0x7f25a806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25ae83d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a8144350 con 0x7f25a815e6d0 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25ae83d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f25a815f1a0 con 0x7f25a8073460 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25ae83d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f25a8072fd0 con 0x7f25a806cf30 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a77fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f25a815e6d0 0x7f25a806c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44922/0 (socket says 172.21.15.106:44922) 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25a8073460 0x7f25a815e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a77fe640 1 -- 172.21.15.106:0/3355584433 learned_addr learned my addr 172.21.15.106:0/3355584433 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:56.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25acdda640 1 --2- 172.21.15.106:0/3355584433 >> v2:172.21.15.134:3300/0 conn(0x7f25a806cf30 0x7f25a806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a6ffd640 1 -- 172.21.15.106:0/3355584433 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3529770302 0 0) 0x7f25a8144350 con 0x7f25a815e6d0 2024-04-25T11:20:56.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a6ffd640 1 -- 172.21.15.106:0/3355584433 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2590003530 con 0x7f25a815e6d0 2024-04-25T11:20:56.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a7fff640 1 -- 172.21.15.106:0/3355584433 >> v2:172.21.15.134:3300/0 conn(0x7f25a806cf30 msgr2=0x7f25a806f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:56.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a7fff640 1 --2- 172.21.15.106:0/3355584433 >> v2:172.21.15.134:3300/0 conn(0x7f25a806cf30 0x7f25a806f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a7fff640 1 -- 172.21.15.106:0/3355584433 >> v1:172.21.15.106:6789/0 conn(0x7f25a815e6d0 legacy=0x7f25a806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.211+0000 7f25a7fff640 1 -- 172.21.15.106:0/3355584433 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a815ebe0 con 0x7f25a8073460 2024-04-25T11:20:56.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25a7fff640 1 --2- 172.21.15.106:0/3355584433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815e190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f25a8072910 tx=0x7f259801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac36df61dc60b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25a6ffd640 1 -- 172.21.15.106:0/3355584433 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980ab650 con 0x7f25a8073460 2024-04-25T11:20:56.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25a6ffd640 1 -- 172.21.15.106:0/3355584433 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25980b0070 con 0x7f25a8073460 2024-04-25T11:20:56.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25a6ffd640 1 -- 172.21.15.106:0/3355584433 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980aba10 con 0x7f25a8073460 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25ae83d640 1 -- 172.21.15.106:0/3355584433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 msgr2=0x7f25a815e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25ae83d640 1 --2- 172.21.15.106:0/3355584433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815e190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f25a8072910 tx=0x7f259801c450 comp rx=0 tx=0).stop 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25ae83d640 1 -- 172.21.15.106:0/3355584433 shutdown_connections 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25ae83d640 1 --2- 172.21.15.106:0/3355584433 >> v2:172.21.15.134:3300/0 conn(0x7f25a806cf30 0x7f25a806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25ae83d640 1 --2- 172.21.15.106:0/3355584433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815e190 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25ae83d640 1 -- 172.21.15.106:0/3355584433 >> 172.21.15.106:0/3355584433 conn(0x7f25a80b3560 msgr2=0x7f25a80b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25ae83d640 1 -- 172.21.15.106:0/3355584433 shutdown_connections 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.215+0000 7f25ae83d640 1 -- 172.21.15.106:0/3355584433 wait complete. 2024-04-25T11:20:56.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 Processor -- start 2024-04-25T11:20:56.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 -- start start 2024-04-25T11:20:56.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25a815e6d0 0x7f25a80b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a813dfe0 0x7f25a813e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25a8070130 con 0x7f25a8073460 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25a8073a80 con 0x7f25a813dfe0 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f25a80702b0 con 0x7f25a815e6d0 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49576/0 (socket says 172.21.15.106:49576) 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 -- 172.21.15.106:0/2825064248 learned_addr learned my addr 172.21.15.106:0/2825064248 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 -- 172.21.15.106:0/2825064248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25a815e6d0 msgr2=0x7f25a80b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25a815e6d0 0x7f25a80b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 -- 172.21.15.106:0/2825064248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a813dfe0 msgr2=0x7f25a813e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a813dfe0 0x7f25a813e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 -- 172.21.15.106:0/2825064248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a813e930 con 0x7f25a8073460 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a7fff640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815d400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2598019060 tx=0x7f259801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a4ff9640 1 -- 172.21.15.106:0/2825064248 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980bf610 con 0x7f25a8073460 2024-04-25T11:20:56.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a4ff9640 1 -- 172.21.15.106:0/2825064248 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25980bf7b0 con 0x7f25a8073460 2024-04-25T11:20:56.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a8140250 con 0x7f25a8073460 2024-04-25T11:20:56.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.219+0000 7f25a4ff9640 1 -- 172.21.15.106:0/2825064248 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980bfb80 con 0x7f25a8073460 2024-04-25T11:20:56.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.223+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a80b5dc0 con 0x7f25a8073460 2024-04-25T11:20:56.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.223+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a806f390 con 0x7f25a8073460 2024-04-25T11:20:56.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.223+0000 7f25a4ff9640 1 -- 172.21.15.106:0/2825064248 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f25980bd020 con 0x7f25a8073460 2024-04-25T11:20:56.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.223+0000 7f25a4ff9640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f259406f430 0x7f2594071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.223+0000 7f25a4ff9640 1 -- 172.21.15.106:0/2825064248 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2598129bf0 con 0x7f25a8073460 2024-04-25T11:20:56.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.223+0000 7f25a77fe640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f259406f430 0x7f2594071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.223+0000 7f25a77fe640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f259406f430 0x7f2594071940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f259c007fd0 tx=0x7f259c007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.227+0000 7f25a4ff9640 1 -- 172.21.15.106:0/2825064248 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25a813e930 con 0x7f25a8073460 2024-04-25T11:20:56.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.399+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f25a81453f0 con 0x7f25a8073460 2024-04-25T11:20:56.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.399+0000 7f25a4ff9640 1 -- 172.21.15.106:0/2825064248 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f2598016070 con 0x7f25a8073460 2024-04-25T11:20:56.400 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T11:20:56.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f259406f430 msgr2=0x7f2594071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f259406f430 0x7f2594071940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f259c007fd0 tx=0x7f259c007900 comp rx=0 tx=0).stop 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 msgr2=0x7f25a815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815d400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2598019060 tx=0x7f259801ac90 comp rx=0 tx=0).stop 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 shutdown_connections 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a813dfe0 0x7f25a813e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25a815e6d0 0x7f25a80b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f259406f430 0x7f2594071940 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 --2- 172.21.15.106:0/2825064248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a8073460 0x7f25a815d400 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 >> 172.21.15.106:0/2825064248 conn(0x7f25a80b3560 msgr2=0x7f25a813b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 shutdown_connections 2024-04-25T11:20:56.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.403+0000 7f25ae83d640 1 -- 172.21.15.106:0/2825064248 wait complete. 2024-04-25T11:20:56.421 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-25T11:20:56.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:56.421 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:56.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[38530]: fuse finished with error 0 and tester_r 0 2024-04-25T11:20:56.433 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:20:56.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:56.434 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:20:56.553 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:20:56.553 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:20:56.553 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:56.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336524k,nr_inodes=4084131,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274692k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16940 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274688k,nr_inodes=818672,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274692k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:20:56.560 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:20:56.560 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:20:56.560 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:56.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16979 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:20:56.566 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:20:56.566 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:20:56.566 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:56.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336524k,nr_inodes=4084131,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274692k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16940 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274688k,nr_inodes=818672,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274692k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:20:56.608 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:56.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2d150a640 1 Processor -- start 2024-04-25T11:20:56.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2d150a640 1 -- start start 2024-04-25T11:20:56.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2d150a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2cc0b4b00 0x7fd2cc15db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2d150a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2cc0b5cc0 con 0x7fd2cc160f10 2024-04-25T11:20:56.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2d150a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2cc0b5f80 con 0x7fd2cc15d6f0 2024-04-25T11:20:56.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2d150a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd2cc0b6260 con 0x7fd2cc0b4b00 2024-04-25T11:20:56.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2caffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd2cc160f10 0x7fd2cc15ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44924/0 (socket says 172.21.15.106:44924) 2024-04-25T11:20:56.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2caffd640 1 -- 172.21.15.106:0/3676067091 learned_addr learned my addr 172.21.15.106:0/3676067091 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:56.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2cb7fe640 1 --2- 172.21.15.106:0/3676067091 >> v2:172.21.15.106:3300/0 conn(0x7fd2cc0b4b00 0x7fd2cc15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2c9ffb640 1 -- 172.21.15.106:0/3676067091 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119644444 0 0) 0x7fd2cc0b5cc0 con 0x7fd2cc160f10 2024-04-25T11:20:56.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2c9ffb640 1 -- 172.21.15.106:0/3676067091 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2b4005740 con 0x7fd2cc160f10 2024-04-25T11:20:56.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2cb7fe640 1 -- 172.21.15.106:0/3676067091 >> v1:172.21.15.134:6789/0 conn(0x7fd2cc15d6f0 legacy=0x7fd2cc0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2cb7fe640 1 -- 172.21.15.106:0/3676067091 >> v1:172.21.15.106:6789/0 conn(0x7fd2cc160f10 legacy=0x7fd2cc15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.763+0000 7fd2cb7fe640 1 -- 172.21.15.106:0/3676067091 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc0b6540 con 0x7fd2cc0b4b00 2024-04-25T11:20:56.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2cb7fe640 1 --2- 172.21.15.106:0/3676067091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc0b4b00 0x7fd2cc15db00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097c0 tx=0x7fd2bc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66091630a4fcec6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2c9ffb640 1 -- 172.21.15.106:0/3676067091 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0b1070 con 0x7fd2cc0b4b00 2024-04-25T11:20:56.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2c9ffb640 1 -- 172.21.15.106:0/3676067091 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0ac620 con 0x7fd2cc0b4b00 2024-04-25T11:20:56.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2c9ffb640 1 -- 172.21.15.106:0/3676067091 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0ac9f0 con 0x7fd2cc0b4b00 2024-04-25T11:20:56.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 -- 172.21.15.106:0/3676067091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc0b4b00 msgr2=0x7fd2cc15db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 --2- 172.21.15.106:0/3676067091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc0b4b00 0x7fd2cc15db00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097c0 tx=0x7fd2bc01c3e0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 -- 172.21.15.106:0/3676067091 shutdown_connections 2024-04-25T11:20:56.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 --2- 172.21.15.106:0/3676067091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc0b4b00 0x7fd2cc15db00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 -- 172.21.15.106:0/3676067091 >> 172.21.15.106:0/3676067091 conn(0x7fd2cc06c7c0 msgr2=0x7fd2cc06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 -- 172.21.15.106:0/3676067091 shutdown_connections 2024-04-25T11:20:56.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 -- 172.21.15.106:0/3676067091 wait complete. 2024-04-25T11:20:56.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 Processor -- start 2024-04-25T11:20:56.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.767+0000 7fd2d150a640 1 -- start start 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2cc0b4b00 0x7fd2cc152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2cc15d6f0 0x7fd2cc146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc160f10 0x7fd2cc147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc072a50 con 0x7fd2cc160f10 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc0d1ba0 con 0x7fd2cc0b4b00 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc072bd0 con 0x7fd2cc15d6f0 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc160f10 0x7fd2cc147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc160f10 0x7fd2cc147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49604/0 (socket says 172.21.15.106:49604) 2024-04-25T11:20:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 -- 172.21.15.106:0/1418994830 learned_addr learned my addr 172.21.15.106:0/1418994830 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2caffd640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2cc0b4b00 0x7fd2cc152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2ca7fc640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2cc15d6f0 0x7fd2cc146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 -- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2cc15d6f0 msgr2=0x7fd2cc146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2cc15d6f0 0x7fd2cc146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 -- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2cc0b4b00 msgr2=0x7fd2cc152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2cc0b4b00 0x7fd2cc152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 -- 172.21.15.106:0/1418994830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc070030 con 0x7fd2cc160f10 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2caffd640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2cc0b4b00 0x7fd2cc152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2cb7fe640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc160f10 0x7fd2cc147370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097c0 tx=0x7fd2bc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2b37fe640 1 -- 172.21.15.106:0/1418994830 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0b1050 con 0x7fd2cc160f10 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2b37fe640 1 -- 172.21.15.106:0/1418994830 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0bd070 con 0x7fd2cc160f10 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc14ef30 con 0x7fd2cc160f10 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2b37fe640 1 -- 172.21.15.106:0/1418994830 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc006e20 con 0x7fd2cc160f10 2024-04-25T11:20:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc14f4f0 con 0x7fd2cc160f10 2024-04-25T11:20:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc1568f0 con 0x7fd2cc160f10 2024-04-25T11:20:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.771+0000 7fd2b37fe640 1 -- 172.21.15.106:0/1418994830 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd2bc0b8670 con 0x7fd2cc160f10 2024-04-25T11:20:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.775+0000 7fd2b37fe640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd2b806f400 0x7fd2b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.775+0000 7fd2caffd640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd2b806f400 0x7fd2b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.775+0000 7fd2b37fe640 1 -- 172.21.15.106:0/1418994830 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd2bc1297d0 con 0x7fd2cc160f10 2024-04-25T11:20:56.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.775+0000 7fd2caffd640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd2b806f400 0x7fd2b8071910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4001570 tx=0x7fd2c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.779+0000 7fd2b37fe640 1 -- 172.21.15.106:0/1418994830 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2bc0c7020 con 0x7fd2cc160f10 2024-04-25T11:20:56.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2cc0d1d20 con 0x7fd2cc160f10 2024-04-25T11:20:56.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2b37fe640 1 -- 172.21.15.106:0/1418994830 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fd2bc016090 con 0x7fd2cc160f10 2024-04-25T11:20:56.959 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:56.959 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3211345151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3211345151},{"type":"v1","addr":"172.21.15.134:6833","nonce":3211345151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1962020849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1962020849},{"type":"v1","addr":"172.21.15.106:6835","nonce":1962020849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3211705687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3211705687},{"type":"v1","addr":"172.21.15.134:6835","nonce":3211705687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:31.811623+0000","modified":"2024-04-25T11:20:34.872100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1141169456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1141169456},{"type":"v1","addr":"172.21.15.106:6837","nonce":1141169456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T11:20:56.959 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T11:20:56.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd2b806f400 msgr2=0x7fd2b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd2b806f400 0x7fd2b8071910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4001570 tx=0x7fd2c4009040 comp rx=0 tx=0).stop 2024-04-25T11:20:56.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc160f10 msgr2=0x7fd2cc147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc160f10 0x7fd2cc147370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097c0 tx=0x7fd2bc01ac20 comp rx=0 tx=0).stop 2024-04-25T11:20:56.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 shutdown_connections 2024-04-25T11:20:56.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd2b806f400 0x7fd2b8071910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2cc160f10 0x7fd2cc147370 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2cc15d6f0 0x7fd2cc146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.959+0000 7fd2d150a640 1 --2- 172.21.15.106:0/1418994830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2cc0b4b00 0x7fd2cc152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.963+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 >> 172.21.15.106:0/1418994830 conn(0x7fd2cc06c7c0 msgr2=0x7fd2cc15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.963+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 shutdown_connections 2024-04-25T11:20:56.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:56.963+0000 7fd2d150a640 1 -- 172.21.15.106:0/1418994830 wait complete. 2024-04-25T11:20:56.979 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:57.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f107a241640 1 Processor -- start 2024-04-25T11:20:57.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f107a241640 1 -- start start 2024-04-25T11:20:57.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f107a241640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1074073460 0x7f10740d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f107a241640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1074073870 con 0x7f10740cef60 2024-04-25T11:20:57.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f107a241640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10740ce050 con 0x7f107415e0a0 2024-04-25T11:20:57.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f107a241640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f107415e6b0 con 0x7f1074073460 2024-04-25T11:20:57.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f1072ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f10740cef60 0x7f10740d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44930/0 (socket says 172.21.15.106:44930) 2024-04-25T11:20:57.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f1073fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1074073460 0x7f10740d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f1072ffd640 1 -- 172.21.15.106:0/592737300 learned_addr learned my addr 172.21.15.106:0/592737300 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f10727fc640 1 -- 172.21.15.106:0/592737300 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345216306 0 0) 0x7f1074073870 con 0x7f10740cef60 2024-04-25T11:20:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f10727fc640 1 -- 172.21.15.106:0/592737300 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f105c003530 con 0x7f10740cef60 2024-04-25T11:20:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f1073fff640 1 -- 172.21.15.106:0/592737300 >> v1:172.21.15.134:6789/0 conn(0x7f107415e0a0 legacy=0x7f10740ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f1073fff640 1 -- 172.21.15.106:0/592737300 >> v1:172.21.15.106:6789/0 conn(0x7f10740cef60 legacy=0x7f10740d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.115+0000 7f1073fff640 1 -- 172.21.15.106:0/592737300 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107415e990 con 0x7f1074073460 2024-04-25T11:20:57.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f1073fff640 1 --2- 172.21.15.106:0/592737300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1074073460 0x7f10740d13f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f106c009d90 tx=0x7f106c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ee2d240ff5e03b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f10727fc640 1 -- 172.21.15.106:0/592737300 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0b2070 con 0x7f1074073460 2024-04-25T11:20:57.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f10727fc640 1 -- 172.21.15.106:0/592737300 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c0a5830 con 0x7f1074073460 2024-04-25T11:20:57.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f10727fc640 1 -- 172.21.15.106:0/592737300 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0a5be0 con 0x7f1074073460 2024-04-25T11:20:57.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f107a241640 1 -- 172.21.15.106:0/592737300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1074073460 msgr2=0x7f10740d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f107a241640 1 --2- 172.21.15.106:0/592737300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1074073460 0x7f10740d13f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f106c009d90 tx=0x7f106c01c970 comp rx=0 tx=0).stop 2024-04-25T11:20:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f107a241640 1 -- 172.21.15.106:0/592737300 shutdown_connections 2024-04-25T11:20:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f107a241640 1 --2- 172.21.15.106:0/592737300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1074073460 0x7f10740d13f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f107a241640 1 -- 172.21.15.106:0/592737300 >> 172.21.15.106:0/592737300 conn(0x7f107406c7c0 msgr2=0x7f107406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f107a241640 1 -- 172.21.15.106:0/592737300 shutdown_connections 2024-04-25T11:20:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.119+0000 7f107a241640 1 -- 172.21.15.106:0/592737300 wait complete. 2024-04-25T11:20:57.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 Processor -- start 2024-04-25T11:20:57.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 -- start start 2024-04-25T11:20:57.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1074073460 0x7f10740d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f107415e0a0 0x7f107415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10740d13f0 0x7f107415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f107415f3f0 con 0x7f10740d13f0 2024-04-25T11:20:57.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f107415f270 con 0x7f107415e0a0 2024-04-25T11:20:57.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f107416bf10 con 0x7f1074073460 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10740d13f0 0x7f107415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10740d13f0 0x7f107415d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49622/0 (socket says 172.21.15.106:49622) 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 -- 172.21.15.106:0/1595722456 learned_addr learned my addr 172.21.15.106:0/1595722456 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f10737fe640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1074073460 0x7f10740d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1072ffd640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f107415e0a0 0x7f107415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 -- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1074073460 msgr2=0x7f10740d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1074073460 0x7f10740d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 -- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f107415e0a0 msgr2=0x7f107415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f107415e0a0 0x7f107415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 -- 172.21.15.106:0/1595722456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10740ab4a0 con 0x7f10740d13f0 2024-04-25T11:20:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1072ffd640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f107415e0a0 0x7f107415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f1073fff640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10740d13f0 0x7f107415d780 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f106c009d90 tx=0x7f106c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f105bfff640 1 -- 172.21.15.106:0/1595722456 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0b2050 con 0x7f10740d13f0 2024-04-25T11:20:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f105bfff640 1 -- 172.21.15.106:0/1595722456 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c0be070 con 0x7f10740d13f0 2024-04-25T11:20:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10740aac00 con 0x7f10740d13f0 2024-04-25T11:20:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f105bfff640 1 -- 172.21.15.106:0/1595722456 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c003db0 con 0x7f10740d13f0 2024-04-25T11:20:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10740ab1c0 con 0x7f10740d13f0 2024-04-25T11:20:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.123+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10740aa970 con 0x7f10740d13f0 2024-04-25T11:20:57.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.127+0000 7f105bfff640 1 -- 172.21.15.106:0/1595722456 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f106c004040 con 0x7f10740d13f0 2024-04-25T11:20:57.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.127+0000 7f105bfff640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f106006f430 0x7f1060071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.127+0000 7f10737fe640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f106006f430 0x7f1060071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.127+0000 7f105bfff640 1 -- 172.21.15.106:0/1595722456 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f106c12a2d0 con 0x7f10740d13f0 2024-04-25T11:20:57.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.127+0000 7f10737fe640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f106006f430 0x7f1060071940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1064007fd0 tx=0x7f1064009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.131+0000 7f105bfff640 1 -- 172.21.15.106:0/1595722456 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f106c016560 con 0x7f10740d13f0 2024-04-25T11:20:57.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.283+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1074143600 con 0x7f10740d13f0 2024-04-25T11:20:57.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f105bfff640 1 -- 172.21.15.106:0/1595722456 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7f106c0f7250 con 0x7f10740d13f0 2024-04-25T11:20:57.285 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:57.285 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:33.406557+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f106006f430 msgr2=0x7f1060071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f106006f430 0x7f1060071940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1064007fd0 tx=0x7f1064009040 comp rx=0 tx=0).stop 2024-04-25T11:20:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10740d13f0 msgr2=0x7f107415d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10740d13f0 0x7f107415d780 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f106c009d90 tx=0x7f106c019710 comp rx=0 tx=0).stop 2024-04-25T11:20:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 shutdown_connections 2024-04-25T11:20:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f106006f430 0x7f1060071940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10740d13f0 0x7f107415d780 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f107415e0a0 0x7f107415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 --2- 172.21.15.106:0/1595722456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1074073460 0x7f10740d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 >> 172.21.15.106:0/1595722456 conn(0x7f107406c7c0 msgr2=0x7f10740cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 shutdown_connections 2024-04-25T11:20:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.287+0000 7f107a241640 1 -- 172.21.15.106:0/1595722456 wait complete. 2024-04-25T11:20:57.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:19:14.141837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:20:29.516176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:20:30.780121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:57.306 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:57.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.459+0000 7ffbd30ea640 1 Processor -- start 2024-04-25T11:20:57.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.459+0000 7ffbd30ea640 1 -- start start 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd30ea640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffbcc143fa0 0x7ffbcc0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd30ea640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffbcc072a20 0x7ffbcc0d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd30ea640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbcc073300 con 0x7ffbcc072e30 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd30ea640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ffbcc0735e0 con 0x7ffbcc143fa0 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd30ea640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffbcc0738c0 con 0x7ffbcc072a20 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd0e86640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffbcc143fa0 0x7ffbcc0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd0e86640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffbcc143fa0 0x7ffbcc0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49630/0 (socket says 172.21.15.106:49630) 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd0e86640 1 -- 172.21.15.106:0/995581765 learned_addr learned my addr 172.21.15.106:0/995581765 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbcbfff640 1 --2- 172.21.15.106:0/995581765 >> v2:172.21.15.134:3300/0 conn(0x7ffbcc072a20 0x7ffbcc0d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd0e86640 1 -- 172.21.15.106:0/995581765 >> v2:172.21.15.134:3300/0 conn(0x7ffbcc072a20 msgr2=0x7ffbcc0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:57.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd0e86640 1 --2- 172.21.15.106:0/995581765 >> v2:172.21.15.134:3300/0 conn(0x7ffbcc072a20 0x7ffbcc0d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd0e86640 1 -- 172.21.15.106:0/995581765 >> v1:172.21.15.134:6789/0 conn(0x7ffbcc072e30 legacy=0x7ffbcc0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:57.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.463+0000 7ffbd0e86640 1 -- 172.21.15.106:0/995581765 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc0b5040 con 0x7ffbcc143fa0 2024-04-25T11:20:57.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd0e86640 1 --2- 172.21.15.106:0/995581765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc0cd930 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0721f0 tx=0x7ffbc001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7634ba8f26b6dbd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbcb7fe640 1 -- 172.21.15.106:0/995581765 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00b1070 con 0x7ffbcc143fa0 2024-04-25T11:20:57.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbcb7fe640 1 -- 172.21.15.106:0/995581765 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc00ac600 con 0x7ffbcc143fa0 2024-04-25T11:20:57.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbcb7fe640 1 -- 172.21.15.106:0/995581765 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00ac9d0 con 0x7ffbcc143fa0 2024-04-25T11:20:57.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 -- 172.21.15.106:0/995581765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 msgr2=0x7ffbcc0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/995581765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc0cd930 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0721f0 tx=0x7ffbc001c4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 -- 172.21.15.106:0/995581765 shutdown_connections 2024-04-25T11:20:57.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/995581765 >> v2:172.21.15.134:3300/0 conn(0x7ffbcc072a20 0x7ffbcc0d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/995581765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc0cd930 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 -- 172.21.15.106:0/995581765 >> 172.21.15.106:0/995581765 conn(0x7ffbcc06c9f0 msgr2=0x7ffbcc06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 -- 172.21.15.106:0/995581765 shutdown_connections 2024-04-25T11:20:57.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 -- 172.21.15.106:0/995581765 wait complete. 2024-04-25T11:20:57.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 Processor -- start 2024-04-25T11:20:57.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.467+0000 7ffbd30ea640 1 -- start start 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbcc072a20 0x7ffbcc1fad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbcc072e30 0x7ffbcc155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc16ad20 con 0x7ffbcc143fa0 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc16aba0 con 0x7ffbcc072e30 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffbcc16aea0 con 0x7ffbcc072a20 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49632/0 (socket says 172.21.15.106:49632) 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 -- 172.21.15.106:0/2153209507 learned_addr learned my addr 172.21.15.106:0/2153209507 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbcbfff640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbcc072e30 0x7ffbcc155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd0e86640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbcc072a20 0x7ffbcc1fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 -- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbcc072a20 msgr2=0x7ffbcc1fad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbcc072a20 0x7ffbcc1fad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 -- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbcc072e30 msgr2=0x7ffbcc155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbcc072e30 0x7ffbcc155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 -- 172.21.15.106:0/2153209507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc1fb650 con 0x7ffbcc143fa0 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbcbfff640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbcc072e30 0x7ffbcc155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd1687640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc145bd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400ba10 tx=0x7ffbc400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbc97fa640 1 -- 172.21.15.106:0/2153209507 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4019070 con 0x7ffbcc143fa0 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbc97fa640 1 -- 172.21.15.106:0/2153209507 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc400cb40 con 0x7ffbcc143fa0 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbc97fa640 1 -- 172.21.15.106:0/2153209507 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4014440 con 0x7ffbcc143fa0 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc1fb930 con 0x7ffbcc143fa0 2024-04-25T11:20:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc1f8970 con 0x7ffbcc143fa0 2024-04-25T11:20:57.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.471+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbcc050750 con 0x7ffbcc143fa0 2024-04-25T11:20:57.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.475+0000 7ffbc97fa640 1 -- 172.21.15.106:0/2153209507 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffbc401c070 con 0x7ffbcc143fa0 2024-04-25T11:20:57.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.475+0000 7ffbc97fa640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffbb806f400 0x7ffbb8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.475+0000 7ffbc97fa640 1 -- 172.21.15.106:0/2153209507 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ffbc408ef90 con 0x7ffbcc143fa0 2024-04-25T11:20:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.475+0000 7ffbd0e86640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffbb806f400 0x7ffbb8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.475+0000 7ffbd0e86640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffbb806f400 0x7ffbb8071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00ae000 tx=0x7ffbc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.479+0000 7ffbc97fa640 1 -- 172.21.15.106:0/2153209507 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffbc405be70 con 0x7ffbcc143fa0 2024-04-25T11:20:57.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.639+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffbcc156730 con 0x7ffbcc143fa0 2024-04-25T11:20:57.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.639+0000 7ffbc97fa640 1 -- 172.21.15.106:0/2153209507 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7ffbc405b770 con 0x7ffbcc143fa0 2024-04-25T11:20:57.639 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T11:20:57.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffbb806f400 msgr2=0x7ffbb8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffbb806f400 0x7ffbb8071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00ae000 tx=0x7ffbc0008040 comp rx=0 tx=0).stop 2024-04-25T11:20:57.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 msgr2=0x7ffbcc145bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc145bd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400ba10 tx=0x7ffbc400bed0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 shutdown_connections 2024-04-25T11:20:57.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffbb806f400 0x7ffbb8071910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbcc143fa0 0x7ffbcc145bd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbcc072e30 0x7ffbcc155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 --2- 172.21.15.106:0/2153209507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbcc072a20 0x7ffbcc1fad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 >> 172.21.15.106:0/2153209507 conn(0x7ffbcc06c9f0 msgr2=0x7ffbcc0cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 shutdown_connections 2024-04-25T11:20:57.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.643+0000 7ffbd30ea640 1 -- 172.21.15.106:0/2153209507 wait complete. 2024-04-25T11:20:57.660 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:57.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f185800b640 1 Processor -- start 2024-04-25T11:20:57.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f185800b640 1 -- start start 2024-04-25T11:20:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f185800b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18500d1e50 0x7f18501644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f185800b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f18500d0cb0 0x7f1850166a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f185800b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185006ec70 con 0x7f18500cf4d0 2024-04-25T11:20:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f185800b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f18500707b0 con 0x7f18500d0cb0 2024-04-25T11:20:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f185800b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f185006f880 con 0x7f18500d1e50 2024-04-25T11:20:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1855da7640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f18500cf4d0 0x7f18500d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44934/0 (socket says 172.21.15.106:44934) 2024-04-25T11:20:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1855da7640 1 -- 172.21.15.106:0/1828456935 learned_addr learned my addr 172.21.15.106:0/1828456935 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f18565a8640 1 --2- 172.21.15.106:0/1828456935 >> v2:172.21.15.134:3300/0 conn(0x7f18500d0cb0 0x7f1850166a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f18555a6640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18500d1e50 0x7f18501644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997828370 0 0) 0x7f185006ec70 con 0x7f18500cf4d0 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f183c003530 con 0x7f18500cf4d0 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 851022310 0 0) 0x7f183c003530 con 0x7f18500cf4d0 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18500d1e50 msgr2=0x7f18501644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18500d1e50 0x7f18501644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 >> v2:172.21.15.134:3300/0 conn(0x7f18500d0cb0 msgr2=0x7f1850166a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 --2- 172.21.15.106:0/1828456935 >> v2:172.21.15.134:3300/0 conn(0x7f18500d0cb0 0x7f1850166a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185006ef50 con 0x7f18500cf4d0 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f184c001680 con 0x7f18500cf4d0 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 >> v1:172.21.15.106:6789/0 conn(0x7f18500cf4d0 legacy=0x7f18500d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0039c0 0x7f183c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f183c004810 0x7f183c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c005160 0x7f183c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00bf10 con 0x7f183c0039c0 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00c090 con 0x7f183c005160 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f183c00c210 con 0x7f183c004810 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1855da7640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0039c0 0x7f183c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f18555a6640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f183c004810 0x7f183c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f18565a8640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c005160 0x7f183c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1855da7640 1 -- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f183c004810 msgr2=0x7f183c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1855da7640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f183c004810 0x7f183c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1855da7640 1 -- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c005160 msgr2=0x7f183c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1855da7640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c005160 0x7f183c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.807+0000 7f1855da7640 1 -- 172.21.15.106:0/1828456935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c003530 con 0x7f183c0039c0 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f1855da7640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0039c0 0x7f183c003e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1850072820 tx=0x7f184c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c48685cfa789e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c007d60 con 0x7f183c0039c0 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f184c0a9020 con 0x7f183c0039c0 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f1854da5640 1 -- 172.21.15.106:0/1828456935 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c005b20 con 0x7f183c0039c0 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f185800b640 1 -- 172.21.15.106:0/1828456935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0039c0 msgr2=0x7f183c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f185800b640 1 --2- 172.21.15.106:0/1828456935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0039c0 0x7f183c003e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1850072820 tx=0x7f184c01cc60 comp rx=0 tx=0).stop 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f18565a8640 1 -- 172.21.15.106:0/1828456935 reap_dead start 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f185800b640 1 -- 172.21.15.106:0/1828456935 shutdown_connections 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f185800b640 1 -- 172.21.15.106:0/1828456935 >> 172.21.15.106:0/1828456935 conn(0x7f185006c9f0 msgr2=0x7f185006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f185800b640 1 -- 172.21.15.106:0/1828456935 shutdown_connections 2024-04-25T11:20:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.811+0000 7f185800b640 1 -- 172.21.15.106:0/1828456935 wait complete. 2024-04-25T11:20:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 Processor -- start 2024-04-25T11:20:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 -- start start 2024-04-25T11:20:57.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c0039c0 0x7f1850166300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850165040 0x7f1850148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1850148690 0x7f185015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f185015ca10 con 0x7f1850165040 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18500d2060 con 0x7f183c0039c0 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f185015cb90 con 0x7f1850148690 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850165040 0x7f1850148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850165040 0x7f1850148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49660/0 (socket says 172.21.15.106:49660) 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 -- 172.21.15.106:0/4154016575 learned_addr learned my addr 172.21.15.106:0/4154016575 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f1855da7640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c0039c0 0x7f1850166300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18565a8640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1850148690 0x7f185015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 -- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1850148690 msgr2=0x7f185015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1850148690 0x7f185015d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 -- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c0039c0 msgr2=0x7f1850166300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c0039c0 0x7f1850166300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 -- 172.21.15.106:0/4154016575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1850154480 con 0x7f1850165040 2024-04-25T11:20:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18565a8640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1850148690 0x7f185015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f1855da7640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c0039c0 0x7f1850166300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18555a6640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850165040 0x7f1850148150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1844011560 tx=0x7f1844011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18427fc640 1 -- 172.21.15.106:0/4154016575 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844015930 con 0x7f1850165040 2024-04-25T11:20:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18427fc640 1 -- 172.21.15.106:0/4154016575 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1844015ad0 con 0x7f1850165040 2024-04-25T11:20:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1850154720 con 0x7f1850165040 2024-04-25T11:20:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f18427fc640 1 -- 172.21.15.106:0/4154016575 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184400f460 con 0x7f1850165040 2024-04-25T11:20:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.815+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501657b0 con 0x7f1850165040 2024-04-25T11:20:57.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.819+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1850050750 con 0x7f1850165040 2024-04-25T11:20:57.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.819+0000 7f18427fc640 1 -- 172.21.15.106:0/4154016575 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f184400f600 con 0x7f1850165040 2024-04-25T11:20:57.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.819+0000 7f18427fc640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f183406f3d0 0x7f18340718e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.819+0000 7f1855da7640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f183406f3d0 0x7f18340718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.819+0000 7f18427fc640 1 -- 172.21.15.106:0/4154016575 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f184408fc30 con 0x7f1850165040 2024-04-25T11:20:57.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.819+0000 7f1855da7640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f183406f3d0 0x7f18340718e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f184c0aa000 tx=0x7f184c000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.823+0000 7f18427fc640 1 -- 172.21.15.106:0/4154016575 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1844019030 con 0x7f1850165040 2024-04-25T11:20:57.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1850152d20 con 0x7f1850165040 2024-04-25T11:20:57.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f18427fc640 1 -- 172.21.15.106:0/4154016575 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f184405cb20 con 0x7f1850165040 2024-04-25T11:20:57.970 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T11:20:57.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f183406f3d0 msgr2=0x7f18340718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f183406f3d0 0x7f18340718e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f184c0aa000 tx=0x7f184c000ff0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850165040 msgr2=0x7f1850148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850165040 0x7f1850148150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1844011560 tx=0x7f1844011a20 comp rx=0 tx=0).stop 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 shutdown_connections 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1850148690 0x7f185015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f183406f3d0 0x7f18340718e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850165040 0x7f1850148150 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 --2- 172.21.15.106:0/4154016575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f183c0039c0 0x7f1850166300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 >> 172.21.15.106:0/4154016575 conn(0x7f185006c9f0 msgr2=0x7f1850153240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 shutdown_connections 2024-04-25T11:20:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:57.971+0000 7f185800b640 1 -- 172.21.15.106:0/4154016575 wait complete. 2024-04-25T11:20:57.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:58.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd35a2db640 1 Processor -- start 2024-04-25T11:20:58.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd35a2db640 1 -- start start 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd35a2db640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540d44a0 0x7fd3540d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd35a2db640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3540d1350 con 0x7fd35415fd10 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd35a2db640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd354163cc0 con 0x7fd3540d2150 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd35a2db640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd354163fa0 con 0x7fd3540d44a0 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd353fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd35415fd10 0x7fd3540b8d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44948/0 (socket says 172.21.15.106:44948) 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd353fff640 1 -- 172.21.15.106:0/3148742888 learned_addr learned my addr 172.21.15.106:0/3148742888 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd358878640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540d44a0 0x7fd3540d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147288336 0 0) 0x7fd3540d1350 con 0x7fd35415fd10 2024-04-25T11:20:58.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd340003530 con 0x7fd35415fd10 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3376176656 0 0) 0x7fd340003530 con 0x7fd35415fd10 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540d44a0 msgr2=0x7fd3540d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540d44a0 0x7fd3540d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 >> v1:172.21.15.134:6789/0 conn(0x7fd3540d2150 legacy=0x7fd3540d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd354164280 con 0x7fd35415fd10 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fd34c001a20 con 0x7fd35415fd10 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 >> v1:172.21.15.106:6789/0 conn(0x7fd35415fd10 legacy=0x7fd3540b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3400039f0 0x7fd340003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540d44a0 0x7fd340004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd340004d50 0x7fd340005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd354163fa0 con 0x7fd3400039f0 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd34000bb30 con 0x7fd340004d50 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd34000bd30 con 0x7fd3540d44a0 2024-04-25T11:20:58.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.139+0000 7fd353fff640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3400039f0 0x7fd340003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd358878640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd340004d50 0x7fd340005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd353fff640 1 -- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540d44a0 msgr2=0x7fd340004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:58.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd353fff640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540d44a0 0x7fd340004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd353fff640 1 -- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd340004d50 msgr2=0x7fd340005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:58.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd353fff640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd340004d50 0x7fd340005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd353fff640 1 -- 172.21.15.106:0/3148742888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd340003530 con 0x7fd3400039f0 2024-04-25T11:20:58.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd358878640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd340004d50 0x7fd340005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:58.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd353fff640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3400039f0 0x7fd340003e60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00a830 tx=0x7fd34c01dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741116f8cbb3ce47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00a480 con 0x7fd3400039f0 2024-04-25T11:20:58.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c0072f0 con 0x7fd3400039f0 2024-04-25T11:20:58.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd352ffd640 1 -- 172.21.15.106:0/3148742888 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0076e0 con 0x7fd3400039f0 2024-04-25T11:20:58.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd35a2db640 1 -- 172.21.15.106:0/3148742888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3400039f0 msgr2=0x7fd340003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd35a2db640 1 --2- 172.21.15.106:0/3148742888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3400039f0 0x7fd340003e60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00a830 tx=0x7fd34c01dce0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd358878640 1 -- 172.21.15.106:0/3148742888 reap_dead start 2024-04-25T11:20:58.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd35a2db640 1 -- 172.21.15.106:0/3148742888 shutdown_connections 2024-04-25T11:20:58.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd35a2db640 1 -- 172.21.15.106:0/3148742888 >> 172.21.15.106:0/3148742888 conn(0x7fd35406d7f0 msgr2=0x7fd35406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:58.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd35a2db640 1 -- 172.21.15.106:0/3148742888 shutdown_connections 2024-04-25T11:20:58.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.143+0000 7fd35a2db640 1 -- 172.21.15.106:0/3148742888 wait complete. 2024-04-25T11:20:58.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 Processor -- start 2024-04-25T11:20:58.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 -- start start 2024-04-25T11:20:58.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3541fcc70 0x7fd354156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3540b51d0 0x7fd354164c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540b55e0 0x7fd354165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd3540b4b20 con 0x7fd3540b51d0 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd3540b49a0 con 0x7fd3541fcc70 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd35416bc80 con 0x7fd3540b55e0 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3540b51d0 0x7fd354164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3540b51d0 0x7fd354164c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49678/0 (socket says 172.21.15.106:49678) 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 -- 172.21.15.106:0/230472601 learned_addr learned my addr 172.21.15.106:0/230472601 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd353fff640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3541fcc70 0x7fd354156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 -- 172.21.15.106:0/230472601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540b55e0 msgr2=0x7fd354165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540b55e0 0x7fd354165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 -- 172.21.15.106:0/230472601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3541fcc70 msgr2=0x7fd354156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3541fcc70 0x7fd354156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 -- 172.21.15.106:0/230472601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3540b6b50 con 0x7fd3540b51d0 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd3537fe640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3540b51d0 0x7fd354164c20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c4d0 tx=0x7fd33c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd350ff9640 1 -- 172.21.15.106:0/230472601 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0052d0 con 0x7fd3540b51d0 2024-04-25T11:20:58.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd350ff9640 1 -- 172.21.15.106:0/230472601 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33c005470 con 0x7fd3540b51d0 2024-04-25T11:20:58.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3540b5ab0 con 0x7fd3540b51d0 2024-04-25T11:20:58.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd350ff9640 1 -- 172.21.15.106:0/230472601 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c012870 con 0x7fd3540b51d0 2024-04-25T11:20:58.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3540b6070 con 0x7fd3540b51d0 2024-04-25T11:20:58.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.147+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd354050750 con 0x7fd3540b51d0 2024-04-25T11:20:58.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.151+0000 7fd350ff9640 1 -- 172.21.15.106:0/230472601 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd33c012a10 con 0x7fd3540b51d0 2024-04-25T11:20:58.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.151+0000 7fd350ff9640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd33406f400 0x7fd334071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.151+0000 7fd353fff640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd33406f400 0x7fd334071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.151+0000 7fd350ff9640 1 -- 172.21.15.106:0/230472601 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd33c08f920 con 0x7fd3540b51d0 2024-04-25T11:20:58.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.151+0000 7fd353fff640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd33406f400 0x7fd334071910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0aa000 tx=0x7fd354164840 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.155+0000 7fd350ff9640 1 -- 172.21.15.106:0/230472601 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd33c05c880 con 0x7fd3540b51d0 2024-04-25T11:20:58.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.311+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd3540d27b0 con 0x7fd3540b51d0 2024-04-25T11:20:58.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.311+0000 7fd350ff9640 1 -- 172.21.15.106:0/230472601 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fd33c05c180 con 0x7fd3540b51d0 2024-04-25T11:20:58.311 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd33406f400 msgr2=0x7fd334071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd33406f400 0x7fd334071910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0aa000 tx=0x7fd354164840 comp rx=0 tx=0).stop 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3540b51d0 msgr2=0x7fd354164c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3540b51d0 0x7fd354164c20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c4d0 tx=0x7fd33c00c990 comp rx=0 tx=0).stop 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 shutdown_connections 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3540b55e0 0x7fd354165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd33406f400 0x7fd334071910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3540b51d0 0x7fd354164c20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 --2- 172.21.15.106:0/230472601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3541fcc70 0x7fd354156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 >> 172.21.15.106:0/230472601 conn(0x7fd35406d7f0 msgr2=0x7fd3540b7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:58.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 shutdown_connections 2024-04-25T11:20:58.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.315+0000 7fd35a2db640 1 -- 172.21.15.106:0/230472601 wait complete. 2024-04-25T11:20:58.332 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:58.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.475+0000 7f2f070c3640 1 Processor -- start 2024-04-25T11:20:58.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.475+0000 7f2f070c3640 1 -- start start 2024-04-25T11:20:58.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f070c3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2f00156f90 0x7f2f000d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f070c3640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f001573a0 con 0x7f2f0006fea0 2024-04-25T11:20:58.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f070c3640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0015d220 con 0x7f2f000b5a80 2024-04-25T11:20:58.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f070c3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2f000d1f90 con 0x7f2f00156f90 2024-04-25T11:20:58.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2ef7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2f0006fea0 0x7f2f000702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44964/0 (socket says 172.21.15.106:44964) 2024-04-25T11:20:58.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f04e5f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2f00156f90 0x7f2f000d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2ef7fff640 1 -- 172.21.15.106:0/1567480871 learned_addr learned my addr 172.21.15.106:0/1567480871 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:58.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f04e5f640 1 -- 172.21.15.106:0/1567480871 >> v1:172.21.15.134:6789/0 conn(0x7f2f000b5a80 legacy=0x7f2f000b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:58.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f04e5f640 1 -- 172.21.15.106:0/1567480871 >> v1:172.21.15.106:6789/0 conn(0x7f2f0006fea0 legacy=0x7f2f000702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f04e5f640 1 -- 172.21.15.106:0/1567480871 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0015d500 con 0x7f2f00156f90 2024-04-25T11:20:58.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2ef77fe640 1 -- 172.21.15.106:0/1567480871 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1660834671 0 0) 0x7f2f001573a0 con 0x7f2f0006fea0 2024-04-25T11:20:58.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.479+0000 7f2f04e5f640 1 --2- 172.21.15.106:0/1567480871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f000d1a50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0097c0 tx=0x7f2efc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d351fe7f0af1f6f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2ef77fe640 1 -- 172.21.15.106:0/1567480871 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc0b0070 con 0x7f2f00156f90 2024-04-25T11:20:58.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2ef77fe640 1 -- 172.21.15.106:0/1567480871 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2efc0ab600 con 0x7f2f00156f90 2024-04-25T11:20:58.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2ef77fe640 1 -- 172.21.15.106:0/1567480871 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc0ab9d0 con 0x7f2f00156f90 2024-04-25T11:20:58.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 -- 172.21.15.106:0/1567480871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 msgr2=0x7f2f000d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 --2- 172.21.15.106:0/1567480871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f000d1a50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0097c0 tx=0x7f2efc01c3d0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 -- 172.21.15.106:0/1567480871 shutdown_connections 2024-04-25T11:20:58.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 --2- 172.21.15.106:0/1567480871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f000d1a50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 -- 172.21.15.106:0/1567480871 >> 172.21.15.106:0/1567480871 conn(0x7f2f0006c7c0 msgr2=0x7f2f0006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:58.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 -- 172.21.15.106:0/1567480871 shutdown_connections 2024-04-25T11:20:58.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 -- 172.21.15.106:0/1567480871 wait complete. 2024-04-25T11:20:58.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 Processor -- start 2024-04-25T11:20:58.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 -- start start 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.483+0000 7f2f070c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f0006fea0 0x7f2f0014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f070c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f000b5a80 0x7f2f00150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f070c3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f001507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f070c3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000b65a0 con 0x7f2f00156f90 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f070c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0015dda0 con 0x7f2f000b5a80 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f070c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2f000b6720 con 0x7f2f0006fea0 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f001507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f001507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49692/0 (socket says 172.21.15.106:49692) 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 -- 172.21.15.106:0/2266230205 learned_addr learned my addr 172.21.15.106:0/2266230205 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2ef7fff640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f000b5a80 0x7f2f00150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f04e5f640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f0006fea0 0x7f2f0014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 -- 172.21.15.106:0/2266230205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f0006fea0 msgr2=0x7f2f0014fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f0006fea0 0x7f2f0014fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 -- 172.21.15.106:0/2266230205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f000b5a80 msgr2=0x7f2f00150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f000b5a80 0x7f2f00150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 -- 172.21.15.106:0/2266230205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f00151070 con 0x7f2f00156f90 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f05660640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f001507c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800ba10 tx=0x7f2ef800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2ef57fa640 1 -- 172.21.15.106:0/2266230205 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8019070 con 0x7f2f00156f90 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2ef57fa640 1 -- 172.21.15.106:0/2266230205 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef800cb40 con 0x7f2f00156f90 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f001546c0 con 0x7f2f00156f90 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2ef57fa640 1 -- 172.21.15.106:0/2266230205 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8014440 con 0x7f2f00156f90 2024-04-25T11:20:58.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f00154960 con 0x7f2f00156f90 2024-04-25T11:20:58.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0015be10 con 0x7f2f00156f90 2024-04-25T11:20:58.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2ef57fa640 1 -- 172.21.15.106:0/2266230205 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ef801c070 con 0x7f2f00156f90 2024-04-25T11:20:58.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2ef57fa640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2eec06f400 0x7f2eec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2ef57fa640 1 -- 172.21.15.106:0/2266230205 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2ef808eff0 con 0x7f2f00156f90 2024-04-25T11:20:58.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f04e5f640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2eec06f400 0x7f2eec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.487+0000 7f2f04e5f640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2eec06f400 0x7f2eec071910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0ad000 tx=0x7f2efc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.491+0000 7f2ef57fa640 1 -- 172.21.15.106:0/2266230205 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ef8023080 con 0x7f2f00156f90 2024-04-25T11:20:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.675+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f00153720 con 0x7f2f00156f90 2024-04-25T11:20:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2ef57fa640 1 -- 172.21.15.106:0/2266230205 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f2ef805c0e0 con 0x7f2f00156f90 2024-04-25T11:20:58.677 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:58.677 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3211345151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3211345151},{"type":"v1","addr":"172.21.15.134:6833","nonce":3211345151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1962020849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1962020849},{"type":"v1","addr":"172.21.15.106:6835","nonce":1962020849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3211705687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3211705687},{"type":"v1","addr":"172.21.15.134:6835","nonce":3211705687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:31.811623+0000","modified":"2024-04-25T11:20:34.872100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1141169456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1141169456},{"type":"v1","addr":"172.21.15.106:6837","nonce":1141169456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T11:20:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T11:20:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2eec06f400 msgr2=0x7f2eec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2eec06f400 0x7f2eec071910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0ad000 tx=0x7f2efc0a6000 comp rx=0 tx=0).stop 2024-04-25T11:20:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 msgr2=0x7f2f001507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f001507c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800ba10 tx=0x7f2ef800bed0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 shutdown_connections 2024-04-25T11:20:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f2eec06f400 0x7f2eec071910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f00156f90 0x7f2f001507c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f000b5a80 0x7f2f00150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 --2- 172.21.15.106:0/2266230205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f0006fea0 0x7f2f0014fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 >> 172.21.15.106:0/2266230205 conn(0x7f2f0006c7c0 msgr2=0x7f2f00075800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 shutdown_connections 2024-04-25T11:20:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.679+0000 7f2f070c3640 1 -- 172.21.15.106:0/2266230205 wait complete. 2024-04-25T11:20:58.698 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:58.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8ba389640 1 Processor -- start 2024-04-25T11:20:58.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8ba389640 1 -- start start 2024-04-25T11:20:58.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8ba389640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8b4073460 0x7ff8b40d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8ba389640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b4073870 con 0x7ff8b415e0a0 2024-04-25T11:20:58.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8ba389640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b40ce050 con 0x7ff8b40cef60 2024-04-25T11:20:58.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8ba389640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b415e6b0 con 0x7ff8b4073460 2024-04-25T11:20:58.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8b8926640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8b4073460 0x7ff8b40d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8b3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff8b415e0a0 0x7ff8b40ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44970/0 (socket says 172.21.15.106:44970) 2024-04-25T11:20:58.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8b3fff640 1 -- 172.21.15.106:0/3097490319 learned_addr learned my addr 172.21.15.106:0/3097490319 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:58.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8b8926640 1 -- 172.21.15.106:0/3097490319 >> v1:172.21.15.134:6789/0 conn(0x7ff8b40cef60 legacy=0x7ff8b40d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8b8926640 1 -- 172.21.15.106:0/3097490319 >> v1:172.21.15.106:6789/0 conn(0x7ff8b415e0a0 legacy=0x7ff8b40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.851+0000 7ff8b8926640 1 -- 172.21.15.106:0/3097490319 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b415e990 con 0x7ff8b4073460 2024-04-25T11:20:58.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8b8926640 1 --2- 172.21.15.106:0/3097490319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b4073460 0x7ff8b40d13f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009d90 tx=0x7ff8ac01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e828941fe12aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8b2ffd640 1 -- 172.21.15.106:0/3097490319 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0b2070 con 0x7ff8b4073460 2024-04-25T11:20:58.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8b2ffd640 1 -- 172.21.15.106:0/3097490319 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0a5830 con 0x7ff8b4073460 2024-04-25T11:20:58.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8b2ffd640 1 -- 172.21.15.106:0/3097490319 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0a5c00 con 0x7ff8b4073460 2024-04-25T11:20:58.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8ba389640 1 -- 172.21.15.106:0/3097490319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b4073460 msgr2=0x7ff8b40d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8ba389640 1 --2- 172.21.15.106:0/3097490319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b4073460 0x7ff8b40d13f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009d90 tx=0x7ff8ac01c970 comp rx=0 tx=0).stop 2024-04-25T11:20:58.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8ba389640 1 -- 172.21.15.106:0/3097490319 shutdown_connections 2024-04-25T11:20:58.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8ba389640 1 --2- 172.21.15.106:0/3097490319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b4073460 0x7ff8b40d13f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8ba389640 1 -- 172.21.15.106:0/3097490319 >> 172.21.15.106:0/3097490319 conn(0x7ff8b406c7c0 msgr2=0x7ff8b406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:58.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8ba389640 1 -- 172.21.15.106:0/3097490319 shutdown_connections 2024-04-25T11:20:58.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.855+0000 7ff8ba389640 1 -- 172.21.15.106:0/3097490319 wait complete. 2024-04-25T11:20:58.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 Processor -- start 2024-04-25T11:20:58.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 -- start start 2024-04-25T11:20:58.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4073460 0x7ff8b415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415e0a0 0x7ff8b4144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b413c490 0x7ff8b413c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b415f3f0 con 0x7ff8b415e0a0 2024-04-25T11:20:58.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b415f270 con 0x7ff8b413c490 2024-04-25T11:20:58.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b416bf10 con 0x7ff8b4073460 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415e0a0 0x7ff8b4144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415e0a0 0x7ff8b4144500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49706/0 (socket says 172.21.15.106:49706) 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 -- 172.21.15.106:0/2517565332 learned_addr learned my addr 172.21.15.106:0/2517565332 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b8926640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b413c490 0x7ff8b413c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b3fff640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4073460 0x7ff8b415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 -- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4073460 msgr2=0x7ff8b415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4073460 0x7ff8b415d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 -- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b413c490 msgr2=0x7ff8b413c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b413c490 0x7ff8b413c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 -- 172.21.15.106:0/2517565332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b413cde0 con 0x7ff8b415e0a0 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b8926640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b413c490 0x7ff8b413c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:58.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b3fff640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4073460 0x7ff8b415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:58.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b37fe640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415e0a0 0x7ff8b4144500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c3f0 tx=0x7ff8a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b0ff9640 1 -- 172.21.15.106:0/2517565332 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8017070 con 0x7ff8b415e0a0 2024-04-25T11:20:58.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b0ff9640 1 -- 172.21.15.106:0/2517565332 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a80052d0 con 0x7ff8b415e0a0 2024-04-25T11:20:58.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b413d7c0 con 0x7ff8b415e0a0 2024-04-25T11:20:58.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8b0ff9640 1 -- 172.21.15.106:0/2517565332 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8003840 con 0x7ff8b415e0a0 2024-04-25T11:20:58.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b4145070 con 0x7ff8b415e0a0 2024-04-25T11:20:58.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.859+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b4144de0 con 0x7ff8b415e0a0 2024-04-25T11:20:58.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.863+0000 7ff8b0ff9640 1 -- 172.21.15.106:0/2517565332 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff8a801a070 con 0x7ff8b415e0a0 2024-04-25T11:20:58.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.863+0000 7ff8b0ff9640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff8a006f400 0x7ff8a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.863+0000 7ff8b0ff9640 1 -- 172.21.15.106:0/2517565332 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff8a808ea50 con 0x7ff8b415e0a0 2024-04-25T11:20:58.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.863+0000 7ff8b3fff640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff8a006f400 0x7ff8a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.863+0000 7ff8b3fff640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff8a006f400 0x7ff8a0071910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4005890 tx=0x7ff8a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:58.867+0000 7ff8b0ff9640 1 -- 172.21.15.106:0/2517565332 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff8a8021020 con 0x7ff8b415e0a0 2024-04-25T11:20:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.015+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8b40b0b20 con 0x7ff8b415e0a0 2024-04-25T11:20:59.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8b0ff9640 1 -- 172.21.15.106:0/2517565332 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7ff8b413cde0 con 0x7ff8b415e0a0 2024-04-25T11:20:59.017 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:59.018 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:33.406557+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:59.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff8a006f400 msgr2=0x7ff8a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff8a006f400 0x7ff8a0071910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4005890 tx=0x7ff8a4009040 comp rx=0 tx=0).stop 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415e0a0 msgr2=0x7ff8b4144500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415e0a0 0x7ff8b4144500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c3f0 tx=0x7ff8a800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 shutdown_connections 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b413c490 0x7ff8b413c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff8a006f400 0x7ff8a0071910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415e0a0 0x7ff8b4144500 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 --2- 172.21.15.106:0/2517565332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4073460 0x7ff8b415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 >> 172.21.15.106:0/2517565332 conn(0x7ff8b406c7c0 msgr2=0x7ff8b40b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 shutdown_connections 2024-04-25T11:20:59.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.019+0000 7ff8ba389640 1 -- 172.21.15.106:0/2517565332 wait complete. 2024-04-25T11:20:59.038 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:20:59.038 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:59.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be7bbe640 1 Processor -- start 2024-04-25T11:20:59.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be7bbe640 1 -- start start 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be7bbe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be00ba970 0x7f6be00d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be7bbe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6be00b5120 0x7f6be00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be7bbe640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be00d0530 con 0x7f6be00d2160 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be7bbe640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6be00d1100 con 0x7f6be00b5120 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be7bbe640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6be00d07d0 con 0x7f6be00ba970 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be5159640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6be00d2160 0x7f6be00d2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44980/0 (socket says 172.21.15.106:44980) 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be5159640 1 -- 172.21.15.106:0/3010572994 learned_addr learned my addr 172.21.15.106:0/3010572994 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be595a640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be00ba970 0x7f6be00d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be615b640 1 --2- 172.21.15.106:0/3010572994 >> v2:172.21.15.134:3300/0 conn(0x7f6be00b5120 0x7f6be00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1367491105 0 0) 0x7f6be00d0530 con 0x7f6be00d2160 2024-04-25T11:20:59.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bcc003530 con 0x7f6be00d2160 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1395050271 0 0) 0x7f6bcc003530 con 0x7f6be00d2160 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be00ba970 msgr2=0x7f6be00d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be00ba970 0x7f6be00d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 >> v2:172.21.15.134:3300/0 conn(0x7f6be00b5120 msgr2=0x7f6be00b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 --2- 172.21.15.106:0/3010572994 >> v2:172.21.15.134:3300/0 conn(0x7f6be00b5120 0x7f6be00b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be00d0ab0 con 0x7f6be00d2160 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f6bd0001800 con 0x7f6be00d2160 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 >> v1:172.21.15.106:6789/0 conn(0x7f6be00d2160 legacy=0x7f6be00d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be00ba970 0x7f6bcc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bcc003f40 0x7f6bcc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc004d50 0x7f6bcc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00d1100 con 0x7f6bcc003f40 2024-04-25T11:20:59.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00d07d0 con 0x7f6bcc004d50 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc00ba20 con 0x7f6be00ba970 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be5159640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bcc003f40 0x7f6bcc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be595a640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be00ba970 0x7f6bcc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be615b640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc004d50 0x7f6bcc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be5159640 1 -- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be00ba970 msgr2=0x7f6bcc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be5159640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be00ba970 0x7f6bcc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be5159640 1 -- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc004d50 msgr2=0x7f6bcc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be5159640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc004d50 0x7f6bcc005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.175+0000 7f6be5159640 1 -- 172.21.15.106:0/3010572994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc003530 con 0x7f6bcc003f40 2024-04-25T11:20:59.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be5159640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bcc003f40 0x7f6bcc004810 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00011e0 tx=0x7f6bd001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935b83ce6ce927e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0007d60 con 0x7f6bcc003f40 2024-04-25T11:20:59.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd0007270 con 0x7f6bcc003f40 2024-04-25T11:20:59.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be4958640 1 -- 172.21.15.106:0/3010572994 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd00057b0 con 0x7f6bcc003f40 2024-04-25T11:20:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be7bbe640 1 -- 172.21.15.106:0/3010572994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bcc003f40 msgr2=0x7f6bcc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be7bbe640 1 --2- 172.21.15.106:0/3010572994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bcc003f40 0x7f6bcc004810 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00011e0 tx=0x7f6bd001f370 comp rx=0 tx=0).stop 2024-04-25T11:20:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be615b640 1 -- 172.21.15.106:0/3010572994 reap_dead start 2024-04-25T11:20:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be7bbe640 1 -- 172.21.15.106:0/3010572994 shutdown_connections 2024-04-25T11:20:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be7bbe640 1 -- 172.21.15.106:0/3010572994 >> 172.21.15.106:0/3010572994 conn(0x7f6be00d57e0 msgr2=0x7f6be00d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be7bbe640 1 -- 172.21.15.106:0/3010572994 shutdown_connections 2024-04-25T11:20:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.179+0000 7f6be7bbe640 1 -- 172.21.15.106:0/3010572994 wait complete. 2024-04-25T11:20:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 Processor -- start 2024-04-25T11:20:59.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 -- start start 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc003f40 0x7f6be015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be01587c0 0x7f6be0161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be0161b60 0x7f6be01f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00b4f20 con 0x7f6be01587c0 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00b4da0 con 0x7f6bcc003f40 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6be00bb080 con 0x7f6be0161b60 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be01587c0 0x7f6be0161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be01587c0 0x7f6be0161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49722/0 (socket says 172.21.15.106:49722) 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 -- 172.21.15.106:0/1838718179 learned_addr learned my addr 172.21.15.106:0/1838718179 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be595a640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc003f40 0x7f6be015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be615b640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be0161b60 0x7f6be01f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 -- 172.21.15.106:0/1838718179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be0161b60 msgr2=0x7f6be01f9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be0161b60 0x7f6be01f9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 -- 172.21.15.106:0/1838718179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc003f40 msgr2=0x7f6be015b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc003f40 0x7f6be015b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 -- 172.21.15.106:0/1838718179 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be015a580 con 0x7f6be01587c0 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be5159640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be01587c0 0x7f6be0161620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0000f00 tx=0x7f6bcc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6bd67fc640 1 -- 172.21.15.106:0/1838718179 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd00070a0 con 0x7f6be01587c0 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6bd67fc640 1 -- 172.21.15.106:0/1838718179 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd0000f80 con 0x7f6be01587c0 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be015a290 con 0x7f6be01587c0 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6bd67fc640 1 -- 172.21.15.106:0/1838718179 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0007660 con 0x7f6be01587c0 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be015ab40 con 0x7f6be01587c0 2024-04-25T11:20:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.183+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be00d2ed0 con 0x7f6be01587c0 2024-04-25T11:20:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.187+0000 7f6bd67fc640 1 -- 172.21.15.106:0/1838718179 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6bd00057b0 con 0x7f6be01587c0 2024-04-25T11:20:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.187+0000 7f6bd67fc640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6bc406f430 0x7f6bc4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.187+0000 7f6be595a640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6bc406f430 0x7f6bc4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.187+0000 7f6bd67fc640 1 -- 172.21.15.106:0/1838718179 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6bd012c290 con 0x7f6be01587c0 2024-04-25T11:20:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.187+0000 7f6be595a640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6bc406f430 0x7f6bc4071940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009160 tx=0x7f6bdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.191+0000 7f6bd67fc640 1 -- 172.21.15.106:0/1838718179 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6bd0019080 con 0x7f6be01587c0 2024-04-25T11:20:59.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.371+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6be00b2c20 con 0x7f6be01587c0 2024-04-25T11:20:59.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.371+0000 7f6bd67fc640 1 -- 172.21.15.106:0/1838718179 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f6bd0007240 con 0x7f6be01587c0 2024-04-25T11:20:59.371 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:59.371 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3211345151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3211345151},{"type":"v1","addr":"172.21.15.134:6833","nonce":3211345151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1962020849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1962020849},{"type":"v1","addr":"172.21.15.106:6835","nonce":1962020849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3211705687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3211705687},{"type":"v1","addr":"172.21.15.134:6835","nonce":3211705687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:31.811623+0000","modified":"2024-04-25T11:20:34.872100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1141169456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1141169456},{"type":"v1","addr":"172.21.15.106:6837","nonce":1141169456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T11:20:59.371 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T11:20:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6bc406f430 msgr2=0x7f6bc4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6bc406f430 0x7f6bc4071940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009160 tx=0x7f6bdc008040 comp rx=0 tx=0).stop 2024-04-25T11:20:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be01587c0 msgr2=0x7f6be0161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be01587c0 0x7f6be0161620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0000f00 tx=0x7f6bcc005840 comp rx=0 tx=0).stop 2024-04-25T11:20:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 shutdown_connections 2024-04-25T11:20:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6be0161b60 0x7f6be01f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6bc406f430 0x7f6bc4071940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be01587c0 0x7f6be0161620 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 --2- 172.21.15.106:0/1838718179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bcc003f40 0x7f6be015b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 >> 172.21.15.106:0/1838718179 conn(0x7f6be00d57e0 msgr2=0x7f6be015fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:59.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 shutdown_connections 2024-04-25T11:20:59.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.375+0000 7f6be7bbe640 1 -- 172.21.15.106:0/1838718179 wait complete. 2024-04-25T11:20:59.391 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb59102e640 1 Processor -- start 2024-04-25T11:20:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb59102e640 1 -- start start 2024-04-25T11:20:59.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb59102e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb58c160f10 0x7fb58c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb59102e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb58c0b4b00 0x7fb58c15db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb59102e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb58c0b5cc0 con 0x7fb58c15d6f0 2024-04-25T11:20:59.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb59102e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb58c15cf50 con 0x7fb58c160f10 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb59102e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb58c0b5f80 con 0x7fb58c0b4b00 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb58c160f10 0x7fb58c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb58c15d6f0 0x7fb58c0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44984/0 (socket says 172.21.15.106:44984) 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58259c640 1 -- 172.21.15.106:0/2253359432 learned_addr learned my addr 172.21.15.106:0/2253359432 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58b59e640 1 --2- 172.21.15.106:0/2253359432 >> v2:172.21.15.134:3300/0 conn(0x7fb58c0b4b00 0x7fb58c15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58ad9d640 1 -- 172.21.15.106:0/2253359432 >> v2:172.21.15.134:3300/0 conn(0x7fb58c0b4b00 msgr2=0x7fb58c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58ad9d640 1 --2- 172.21.15.106:0/2253359432 >> v2:172.21.15.134:3300/0 conn(0x7fb58c0b4b00 0x7fb58c15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58ad9d640 1 -- 172.21.15.106:0/2253359432 >> v1:172.21.15.106:6789/0 conn(0x7fb58c15d6f0 legacy=0x7fb58c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58a59c640 1 -- 172.21.15.106:0/2253359432 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114662396 0 0) 0x7fb58c0b5cc0 con 0x7fb58c15d6f0 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58ad9d640 1 -- 172.21.15.106:0/2253359432 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c0b6260 con 0x7fb58c160f10 2024-04-25T11:20:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.535+0000 7fb58b59e640 1 --2- 172.21.15.106:0/2253359432 >> v2:172.21.15.134:3300/0 conn(0x7fb58c0b4b00 0x7fb58c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb58ad9d640 1 --2- 172.21.15.106:0/2253359432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c160f10 0x7fb58c15ca10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0ce290 tx=0x7fb57c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f167a1bfcfc01a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb58a59c640 1 -- 172.21.15.106:0/2253359432 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0b1070 con 0x7fb58c160f10 2024-04-25T11:20:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb58a59c640 1 -- 172.21.15.106:0/2253359432 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57c0ac620 con 0x7fb58c160f10 2024-04-25T11:20:59.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb58a59c640 1 -- 172.21.15.106:0/2253359432 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0ac9f0 con 0x7fb58c160f10 2024-04-25T11:20:59.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb59102e640 1 -- 172.21.15.106:0/2253359432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c160f10 msgr2=0x7fb58c15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb59102e640 1 --2- 172.21.15.106:0/2253359432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c160f10 0x7fb58c15ca10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0ce290 tx=0x7fb57c01c3c0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb59102e640 1 -- 172.21.15.106:0/2253359432 shutdown_connections 2024-04-25T11:20:59.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb59102e640 1 --2- 172.21.15.106:0/2253359432 >> v2:172.21.15.134:3300/0 conn(0x7fb58c0b4b00 0x7fb58c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb59102e640 1 --2- 172.21.15.106:0/2253359432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c160f10 0x7fb58c15ca10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb59102e640 1 -- 172.21.15.106:0/2253359432 >> 172.21.15.106:0/2253359432 conn(0x7fb58c06c7c0 msgr2=0x7fb58c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:59.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb59102e640 1 -- 172.21.15.106:0/2253359432 shutdown_connections 2024-04-25T11:20:59.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.539+0000 7fb59102e640 1 -- 172.21.15.106:0/2253359432 wait complete. 2024-04-25T11:20:59.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 Processor -- start 2024-04-25T11:20:59.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 -- start start 2024-04-25T11:20:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c0b4b00 0x7fb58c13b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c15d6f0 0x7fb58c14f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c160f10 0x7fb58c14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c0d1b30 con 0x7fb58c0b4b00 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c0d19b0 con 0x7fb58c15d6f0 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb58c072a50 con 0x7fb58c160f10 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c0b4b00 0x7fb58c13b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c0b4b00 0x7fb58c13b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49734/0 (socket says 172.21.15.106:49734) 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 -- 172.21.15.106:0/223220474 learned_addr learned my addr 172.21.15.106:0/223220474 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58b59e640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c160f10 0x7fb58c14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58259c640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c15d6f0 0x7fb58c14f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 -- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c160f10 msgr2=0x7fb58c14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c160f10 0x7fb58c14f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 -- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c15d6f0 msgr2=0x7fb58c14f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c15d6f0 0x7fb58c14f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 -- 172.21.15.106:0/223220474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c156750 con 0x7fb58c0b4b00 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58b59e640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c160f10 0x7fb58c14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb58ad9d640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c0b4b00 0x7fb58c13b9a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb57c018fd0 tx=0x7fb57c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb583fff640 1 -- 172.21.15.106:0/223220474 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0b1050 con 0x7fb58c0b4b00 2024-04-25T11:20:59.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb583fff640 1 -- 172.21.15.106:0/223220474 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57c0b2040 con 0x7fb58c0b4b00 2024-04-25T11:20:59.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58c156a30 con 0x7fb58c0b4b00 2024-04-25T11:20:59.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb583fff640 1 -- 172.21.15.106:0/223220474 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c004220 con 0x7fb58c0b4b00 2024-04-25T11:20:59.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58c156ff0 con 0x7fb58c0b4b00 2024-04-25T11:20:59.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.543+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58c0b3fc0 con 0x7fb58c0b4b00 2024-04-25T11:20:59.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.547+0000 7fb583fff640 1 -- 172.21.15.106:0/223220474 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb57c0bf060 con 0x7fb58c0b4b00 2024-04-25T11:20:59.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.547+0000 7fb583fff640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb56c06f400 0x7fb56c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.547+0000 7fb583fff640 1 -- 172.21.15.106:0/223220474 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb57c128da0 con 0x7fb58c0b4b00 2024-04-25T11:20:59.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.547+0000 7fb58259c640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb56c06f400 0x7fb56c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.547+0000 7fb58259c640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb56c06f400 0x7fb56c071910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb58c1556b0 tx=0x7fb574009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.551+0000 7fb583fff640 1 -- 172.21.15.106:0/223220474 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb57c016070 con 0x7fb58c0b4b00 2024-04-25T11:20:59.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.739+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb58c06da30 con 0x7fb58c0b4b00 2024-04-25T11:20:59.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb583fff640 1 -- 172.21.15.106:0/223220474 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fb57c0f5c90 con 0x7fb58c0b4b00 2024-04-25T11:20:59.741 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:20:59.741 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3211345151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3211345151},{"type":"v1","addr":"172.21.15.134:6833","nonce":3211345151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1962020849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1962020849},{"type":"v1","addr":"172.21.15.106:6835","nonce":1962020849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3211705687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3211705687},{"type":"v1","addr":"172.21.15.134:6835","nonce":3211705687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:31.811623+0000","modified":"2024-04-25T11:20:34.872100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1141169456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1141169456},{"type":"v1","addr":"172.21.15.106:6837","nonce":1141169456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T11:20:59.741 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T11:20:59.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb56c06f400 msgr2=0x7fb56c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb56c06f400 0x7fb56c071910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb58c1556b0 tx=0x7fb574009040 comp rx=0 tx=0).stop 2024-04-25T11:20:59.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c0b4b00 msgr2=0x7fb58c13b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c0b4b00 0x7fb58c13b9a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb57c018fd0 tx=0x7fb57c01ac00 comp rx=0 tx=0).stop 2024-04-25T11:20:59.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 shutdown_connections 2024-04-25T11:20:59.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c160f10 0x7fb58c14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c15d6f0 0x7fb58c14f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb56c06f400 0x7fb56c071910 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 --2- 172.21.15.106:0/223220474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb58c0b4b00 0x7fb58c13b9a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 >> 172.21.15.106:0/223220474 conn(0x7fb58c06c7c0 msgr2=0x7fb58c15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:59.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 shutdown_connections 2024-04-25T11:20:59.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.743+0000 7fb59102e640 1 -- 172.21.15.106:0/223220474 wait complete. 2024-04-25T11:20:59.760 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.907+0000 7f702b3c3640 1 Processor -- start 2024-04-25T11:20:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.907+0000 7f702b3c3640 1 -- start start 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f702b3c3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7024160f10 0x7f702415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f702b3c3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70240b4b00 0x7f702415db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f702b3c3640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70240b5cc0 con 0x7f702415d6f0 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f702b3c3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f702415cf50 con 0x7f70240b4b00 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f702b3c3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f70240b5f80 con 0x7f7024160f10 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f7029960640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70240b4b00 0x7f702415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f7029960640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70240b4b00 0x7f702415db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49738/0 (socket says 172.21.15.106:49738) 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f7029960640 1 -- 172.21.15.106:0/675314682 learned_addr learned my addr 172.21.15.106:0/675314682 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f702915f640 1 --2- 172.21.15.106:0/675314682 >> v2:172.21.15.134:3300/0 conn(0x7f7024160f10 0x7f702415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f7029960640 1 -- 172.21.15.106:0/675314682 >> v2:172.21.15.134:3300/0 conn(0x7f7024160f10 msgr2=0x7f702415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f7029960640 1 --2- 172.21.15.106:0/675314682 >> v2:172.21.15.134:3300/0 conn(0x7f7024160f10 0x7f702415ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f7029960640 1 -- 172.21.15.106:0/675314682 >> v1:172.21.15.134:6789/0 conn(0x7f702415d6f0 legacy=0x7f70240b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.911+0000 7f7029960640 1 -- 172.21.15.106:0/675314682 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70240b6260 con 0x7f70240b4b00 2024-04-25T11:20:59.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f7029960640 1 --2- 172.21.15.106:0/675314682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70240b4b00 0x7f702415db00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7024072220 tx=0x7f701c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a20b17aa85eedd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f701bfff640 1 -- 172.21.15.106:0/675314682 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0a57f0 con 0x7f70240b4b00 2024-04-25T11:20:59.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f701bfff640 1 -- 172.21.15.106:0/675314682 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701c0a5990 con 0x7f70240b4b00 2024-04-25T11:20:59.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f701bfff640 1 -- 172.21.15.106:0/675314682 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0a5d60 con 0x7f70240b4b00 2024-04-25T11:20:59.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 -- 172.21.15.106:0/675314682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70240b4b00 msgr2=0x7f702415db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 --2- 172.21.15.106:0/675314682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70240b4b00 0x7f702415db00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7024072220 tx=0x7f701c01c930 comp rx=0 tx=0).stop 2024-04-25T11:20:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 -- 172.21.15.106:0/675314682 shutdown_connections 2024-04-25T11:20:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 --2- 172.21.15.106:0/675314682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70240b4b00 0x7f702415db00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 --2- 172.21.15.106:0/675314682 >> v2:172.21.15.134:3300/0 conn(0x7f7024160f10 0x7f702415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 -- 172.21.15.106:0/675314682 >> 172.21.15.106:0/675314682 conn(0x7f702406c7c0 msgr2=0x7f702406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 -- 172.21.15.106:0/675314682 shutdown_connections 2024-04-25T11:20:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 -- 172.21.15.106:0/675314682 wait complete. 2024-04-25T11:20:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.915+0000 7f702b3c3640 1 Processor -- start 2024-04-25T11:20:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 -- start start 2024-04-25T11:20:59.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240b4b00 0x7f70240b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f702415d6f0 0x7f7024146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7024160f10 0x7f7024072a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70240d1b10 con 0x7f7024160f10 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70240d1990 con 0x7f70240b4b00 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7024169690 con 0x7f702415d6f0 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7024160f10 0x7f7024072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7024160f10 0x7f7024072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49754/0 (socket says 172.21.15.106:49754) 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 -- 172.21.15.106:0/2260821868 learned_addr learned my addr 172.21.15.106:0/2260821868 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702915f640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240b4b00 0x7f70240b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702895e640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f702415d6f0 0x7f7024146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 -- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f702415d6f0 msgr2=0x7f7024146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f702415d6f0 0x7f7024146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 -- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240b4b00 msgr2=0x7f70240b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240b4b00 0x7f70240b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 -- 172.21.15.106:0/2260821868 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7024154ad0 con 0x7f7024160f10 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702915f640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240b4b00 0x7f70240b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7029960640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7024160f10 0x7f7024072a50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f701c0af040 tx=0x7f701c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7019ffb640 1 -- 172.21.15.106:0/2260821868 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c009510 con 0x7f7024160f10 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7019ffb640 1 -- 172.21.15.106:0/2260821868 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701c0be070 con 0x7f7024160f10 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f7019ffb640 1 -- 172.21.15.106:0/2260821868 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0b88b0 con 0x7f7024160f10 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7024154db0 con 0x7f7024160f10 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7024155370 con 0x7f7024160f10 2024-04-25T11:20:59.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.919+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702415df80 con 0x7f7024160f10 2024-04-25T11:20:59.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.923+0000 7f7019ffb640 1 -- 172.21.15.106:0/2260821868 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f701c0b8a50 con 0x7f7024160f10 2024-04-25T11:20:59.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.923+0000 7f7019ffb640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f700806f400 0x7f7008071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.923+0000 7f7019ffb640 1 -- 172.21.15.106:0/2260821868 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f701c12a3f0 con 0x7f7024160f10 2024-04-25T11:20:59.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.923+0000 7f702915f640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f700806f400 0x7f7008071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.923+0000 7f702915f640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f700806f400 0x7f7008071910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70240d2420 tx=0x7f7020008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:20:59.927+0000 7f7019ffb640 1 -- 172.21.15.106:0/2260821868 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f701c0165e0 con 0x7f7024160f10 2024-04-25T11:21:00.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.087+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f702415dca0 con 0x7f7024160f10 2024-04-25T11:21:00.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f7019ffb640 1 -- 172.21.15.106:0/2260821868 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7f701c0f72e0 con 0x7f7024160f10 2024-04-25T11:21:00.089 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:21:00.089 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:20:33.406557+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:29.516176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:30.780121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:00.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f700806f400 msgr2=0x7f7008071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f700806f400 0x7f7008071910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70240d2420 tx=0x7f7020008040 comp rx=0 tx=0).stop 2024-04-25T11:21:00.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7024160f10 msgr2=0x7f7024072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7024160f10 0x7f7024072a50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f701c0af040 tx=0x7f701c0196d0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 shutdown_connections 2024-04-25T11:21:00.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f700806f400 0x7f7008071910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7024160f10 0x7f7024072a50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f702415d6f0 0x7f7024146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 --2- 172.21.15.106:0/2260821868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240b4b00 0x7f70240b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 >> 172.21.15.106:0/2260821868 conn(0x7f702406c7c0 msgr2=0x7f702414bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:00.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 shutdown_connections 2024-04-25T11:21:00.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.091+0000 7f702b3c3640 1 -- 172.21.15.106:0/2260821868 wait complete. 2024-04-25T11:21:00.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:21:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f132053f640 1 Processor -- start 2024-04-25T11:21:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f132053f640 1 -- start start 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f132053f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d1b10 0x7f13180d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f132053f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f131806dfa0 0x7f131806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f132053f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1318164230 0x7f1318164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f132053f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f13180d0f90 con 0x7f131806dfa0 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f132053f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f131806e8f0 con 0x7f1318164230 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f132053f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13180d2850 con 0x7f13180d1b10 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131dada640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f131806dfa0 0x7f131806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131dada640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f131806dfa0 0x7f131806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49758/0 (socket says 172.21.15.106:49758) 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131dada640 1 -- 172.21.15.106:0/394678527 learned_addr learned my addr 172.21.15.106:0/394678527 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131eadc640 1 --2- 172.21.15.106:0/394678527 >> v2:172.21.15.134:3300/0 conn(0x7f1318164230 0x7f1318164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131e2db640 1 --2- 172.21.15.106:0/394678527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d1b10 0x7f13180d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131dada640 1 -- 172.21.15.106:0/394678527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d1b10 msgr2=0x7f13180d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131dada640 1 --2- 172.21.15.106:0/394678527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d1b10 0x7f13180d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131dada640 1 -- 172.21.15.106:0/394678527 >> v2:172.21.15.134:3300/0 conn(0x7f1318164230 msgr2=0x7f1318164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131dada640 1 --2- 172.21.15.106:0/394678527 >> v2:172.21.15.134:3300/0 conn(0x7f1318164230 0x7f1318164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.263+0000 7f131dada640 1 -- 172.21.15.106:0/394678527 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13180cf7c0 con 0x7f131806dfa0 2024-04-25T11:21:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f131dada640 1 --2- 172.21.15.106:0/394678527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131806e3b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f13180cdac0 tx=0x7f130c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87daee3acd33234e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f131d2d9640 1 -- 172.21.15.106:0/394678527 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0b0070 con 0x7f131806dfa0 2024-04-25T11:21:00.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f131d2d9640 1 -- 172.21.15.106:0/394678527 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f130c0ab600 con 0x7f131806dfa0 2024-04-25T11:21:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f131d2d9640 1 -- 172.21.15.106:0/394678527 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0ab9d0 con 0x7f131806dfa0 2024-04-25T11:21:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 -- 172.21.15.106:0/394678527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 msgr2=0x7f131806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 --2- 172.21.15.106:0/394678527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131806e3b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f13180cdac0 tx=0x7f130c01c490 comp rx=0 tx=0).stop 2024-04-25T11:21:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 -- 172.21.15.106:0/394678527 shutdown_connections 2024-04-25T11:21:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 --2- 172.21.15.106:0/394678527 >> v2:172.21.15.134:3300/0 conn(0x7f1318164230 0x7f1318164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 --2- 172.21.15.106:0/394678527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131806e3b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 --2- 172.21.15.106:0/394678527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13180d1b10 0x7f13180d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 -- 172.21.15.106:0/394678527 >> 172.21.15.106:0/394678527 conn(0x7f13180cfff0 msgr2=0x7f13180d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 -- 172.21.15.106:0/394678527 shutdown_connections 2024-04-25T11:21:00.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.267+0000 7f132053f640 1 -- 172.21.15.106:0/394678527 wait complete. 2024-04-25T11:21:00.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 Processor -- start 2024-04-25T11:21:00.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 -- start start 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1b10 0x7f131815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1318164230 0x7f13180a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13180cfc40 con 0x7f131806dfa0 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1318164f90 con 0x7f13180d1b10 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13180cfdc0 con 0x7f1318164230 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131815dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49774/0 (socket says 172.21.15.106:49774) 2024-04-25T11:21:00.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 -- 172.21.15.106:0/79355796 learned_addr learned my addr 172.21.15.106:0/79355796 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131dada640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1b10 0x7f131815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131eadc640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1318164230 0x7f13180a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 -- 172.21.15.106:0/79355796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1318164230 msgr2=0x7f13180a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1318164230 0x7f13180a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 -- 172.21.15.106:0/79355796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1b10 msgr2=0x7f131815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1b10 0x7f131815b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 -- 172.21.15.106:0/79355796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131815e1f0 con 0x7f131806dfa0 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f131e2db640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131815dcb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f131400d800 tx=0x7f131400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f130affd640 1 -- 172.21.15.106:0/79355796 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13140096a0 con 0x7f131806dfa0 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f130affd640 1 -- 172.21.15.106:0/79355796 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1314019070 con 0x7f131806dfa0 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131815bf10 con 0x7f131806dfa0 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f130affd640 1 -- 172.21.15.106:0/79355796 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314009a80 con 0x7f131806dfa0 2024-04-25T11:21:00.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.271+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131815c4d0 con 0x7f131806dfa0 2024-04-25T11:21:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.275+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1318050750 con 0x7f131806dfa0 2024-04-25T11:21:00.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.275+0000 7f130affd640 1 -- 172.21.15.106:0/79355796 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1314009c20 con 0x7f131806dfa0 2024-04-25T11:21:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.275+0000 7f130affd640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12fc06f430 0x7f12fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.275+0000 7f130affd640 1 -- 172.21.15.106:0/79355796 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f131408f5f0 con 0x7f131806dfa0 2024-04-25T11:21:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.275+0000 7f131dada640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12fc06f430 0x7f12fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.275+0000 7f131dada640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12fc06f430 0x7f12fc071940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f130c0ad000 tx=0x7f130c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:00.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.283+0000 7f130affd640 1 -- 172.21.15.106:0/79355796 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f131405c570 con 0x7f131806dfa0 2024-04-25T11:21:00.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.479+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f131815cb90 con 0x7f131806dfa0 2024-04-25T11:21:00.479 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-25T11:21:00.479+0000 7f8fc7653640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:21:00.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.483+0000 7f130affd640 1 -- 172.21.15.106:0/79355796 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f131405be70 con 0x7f131806dfa0 2024-04-25T11:21:00.482 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:21:00.486 INFO:tasks.ceph.mds.c.smithi106.stderr: -2531> 2024-04-25T11:20:36.379+0000 7f872eeb7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12fc06f430 msgr2=0x7f12fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12fc06f430 0x7f12fc071940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f130c0ad000 tx=0x7f130c008040 comp rx=0 tx=0).stop 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 msgr2=0x7f131815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131815dcb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f131400d800 tx=0x7f131400dcc0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 shutdown_connections 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1318164230 0x7f13180a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13180d1b10 0x7f131815b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f12fc06f430 0x7f12fc071940 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 --2- 172.21.15.106:0/79355796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f131806dfa0 0x7f131815dcb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 >> 172.21.15.106:0/79355796 conn(0x7f13180cfff0 msgr2=0x7f13180b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 shutdown_connections 2024-04-25T11:21:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.491+0000 7f132053f640 1 -- 172.21.15.106:0/79355796 wait complete. 2024-04-25T11:21:00.510 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:21:00.547 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-25T11:21:00.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.647+0000 7f7932896640 1 Processor -- start 2024-04-25T11:21:00.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.647+0000 7f7932896640 1 -- start start 2024-04-25T11:21:00.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f7932896640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f792c0cf4d0 0x7f792c0d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f7932896640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0d0cb0 0x7f792c0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f7932896640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792c06ec70 con 0x7f792c15ca10 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f7932896640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f792c0707b0 con 0x7f792c0cf4d0 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f7932896640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f792c06f880 con 0x7f792c0d0cb0 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f792c0cf4d0 0x7f792c0d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f792c0cf4d0 0x7f792c0d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49806/0 (socket says 172.21.15.106:49806) 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792bfff640 1 -- 172.21.15.106:0/367655958 learned_addr learned my addr 172.21.15.106:0/367655958 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792b7fe640 1 --2- 172.21.15.106:0/367655958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0d0cb0 0x7f792c0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792bfff640 1 -- 172.21.15.106:0/367655958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0d0cb0 msgr2=0x7f792c0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792bfff640 1 --2- 172.21.15.106:0/367655958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0d0cb0 0x7f792c0d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792bfff640 1 -- 172.21.15.106:0/367655958 >> v1:172.21.15.134:6789/0 conn(0x7f792c15ca10 legacy=0x7f792c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:00.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792bfff640 1 -- 172.21.15.106:0/367655958 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c06ef50 con 0x7f792c0cf4d0 2024-04-25T11:21:00.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792b7fe640 1 --2- 172.21.15.106:0/367655958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0d0cb0 0x7f792c0d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:00.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.651+0000 7f792bfff640 1 --2- 172.21.15.106:0/367655958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf4d0 0x7f792c0d1740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7920009870 tx=0x7f792001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56807922bf214c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:00.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f792affd640 1 -- 172.21.15.106:0/367655958 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200b0070 con 0x7f792c0cf4d0 2024-04-25T11:21:00.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f792affd640 1 -- 172.21.15.106:0/367655958 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79200ab600 con 0x7f792c0cf4d0 2024-04-25T11:21:00.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f792affd640 1 -- 172.21.15.106:0/367655958 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200ab9d0 con 0x7f792c0cf4d0 2024-04-25T11:21:00.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- 172.21.15.106:0/367655958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf4d0 msgr2=0x7f792c0d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 --2- 172.21.15.106:0/367655958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf4d0 0x7f792c0d1740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7920009870 tx=0x7f792001c490 comp rx=0 tx=0).stop 2024-04-25T11:21:00.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- 172.21.15.106:0/367655958 shutdown_connections 2024-04-25T11:21:00.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 --2- 172.21.15.106:0/367655958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0d0cb0 0x7f792c0d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 --2- 172.21.15.106:0/367655958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf4d0 0x7f792c0d1740 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- 172.21.15.106:0/367655958 >> 172.21.15.106:0/367655958 conn(0x7f792c06c9f0 msgr2=0x7f792c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:00.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- 172.21.15.106:0/367655958 shutdown_connections 2024-04-25T11:21:00.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- 172.21.15.106:0/367655958 wait complete. 2024-04-25T11:21:00.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 Processor -- start 2024-04-25T11:21:00.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- start start 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0cf4d0 0x7f792c149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0d0cb0 0x7f792c136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c15ca10 0x7f792c137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f792c167880 con 0x7f792c0d0cb0 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f792c167700 con 0x7f792c15ca10 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.655+0000 7f7932896640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f792c167a00 con 0x7f792c0cf4d0 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0d0cb0 0x7f792c136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0d0cb0 0x7f792c136ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49808/0 (socket says 172.21.15.106:49808) 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 -- 172.21.15.106:0/1672126150 learned_addr learned my addr 172.21.15.106:0/1672126150 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:00.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792bfff640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0cf4d0 0x7f792c149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7930e33640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c15ca10 0x7f792c137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 -- 172.21.15.106:0/1672126150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0cf4d0 msgr2=0x7f792c149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0cf4d0 0x7f792c149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 -- 172.21.15.106:0/1672126150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c15ca10 msgr2=0x7f792c137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c15ca10 0x7f792c137220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 -- 172.21.15.106:0/1672126150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c135c40 con 0x7f792c0d0cb0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792b7fe640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0d0cb0 0x7f792c136ce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c820 tx=0x7f791c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7928ff9640 1 -- 172.21.15.106:0/1672126150 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c015070 con 0x7f792c0d0cb0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7928ff9640 1 -- 172.21.15.106:0/1672126150 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791c007a30 con 0x7f792c0d0cb0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c135f20 con 0x7f792c0d0cb0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7928ff9640 1 -- 172.21.15.106:0/1672126150 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c007e00 con 0x7f792c0d0cb0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c1364e0 con 0x7f792c0d0cb0 2024-04-25T11:21:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c050750 con 0x7f792c0d0cb0 2024-04-25T11:21:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7928ff9640 1 -- 172.21.15.106:0/1672126150 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f791c018070 con 0x7f792c0d0cb0 2024-04-25T11:21:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7928ff9640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f791806f400 0x7f7918071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792bfff640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f791806f400 0x7f7918071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f7928ff9640 1 -- 172.21.15.106:0/1672126150 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f791c08f720 con 0x7f792c0d0cb0 2024-04-25T11:21:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.659+0000 7f792bfff640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f791806f400 0x7f7918071910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79200ad000 tx=0x7f79200a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:00.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.663+0000 7f7928ff9640 1 -- 172.21.15.106:0/1672126150 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f791c05c690 con 0x7f792c0d0cb0 2024-04-25T11:21:00.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:00.843+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f792c0cd5e0 con 0x7f792c0d0cb0 2024-04-25T11:21:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7928ff9640 1 -- 172.21.15.106:0/1672126150 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f791c05bf90 con 0x7f792c0d0cb0 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f791806f400 msgr2=0x7f7918071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f791806f400 0x7f7918071910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79200ad000 tx=0x7f79200a6000 comp rx=0 tx=0).stop 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0d0cb0 msgr2=0x7f792c136ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0d0cb0 0x7f792c136ce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c820 tx=0x7f791c00cce0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 shutdown_connections 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c15ca10 0x7f792c137220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f791806f400 0x7f7918071910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0d0cb0 0x7f792c136ce0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 --2- 172.21.15.106:0/1672126150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c0cf4d0 0x7f792c149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 >> 172.21.15.106:0/1672126150 conn(0x7f792c06c9f0 msgr2=0x7f792c06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 shutdown_connections 2024-04-25T11:21:01.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.487+0000 7f7932896640 1 -- 172.21.15.106:0/1672126150 wait complete. 2024-04-25T11:21:01.514 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:21:01.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.711+0000 7f418062e640 1 Processor -- start 2024-04-25T11:21:01.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.711+0000 7f418062e640 1 -- start start 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f418062e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780b6ea0 0x7f41780b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f418062e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f41780b77f0 0x7f41780b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f418062e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41780b8140 con 0x7f417806daf0 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f418062e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f41780b8b70 con 0x7f41780b77f0 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f418062e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f417809cf70 con 0x7f41780b6ea0 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417e3ca640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f417806daf0 0x7f417806e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45002/0 (socket says 172.21.15.106:45002) 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417e3ca640 1 -- 172.21.15.106:0/3329953284 learned_addr learned my addr 172.21.15.106:0/3329953284 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417ebcb640 1 --2- 172.21.15.106:0/3329953284 >> v2:172.21.15.134:3300/0 conn(0x7f41780b77f0 0x7f41780b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417dbc9640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780b6ea0 0x7f41780b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1907371450 0 0) 0x7f41780b8140 con 0x7f417806daf0 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4160003530 con 0x7f417806daf0 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 750882045 0 0) 0x7f4160003530 con 0x7f417806daf0 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780b6ea0 msgr2=0x7f41780b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780b6ea0 0x7f41780b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 >> v2:172.21.15.134:3300/0 conn(0x7f41780b77f0 msgr2=0x7f41780b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 --2- 172.21.15.106:0/3329953284 >> v2:172.21.15.134:3300/0 conn(0x7f41780b77f0 0x7f41780b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41780d29c0 con 0x7f417806daf0 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f4170001800 con 0x7f417806daf0 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 >> v1:172.21.15.106:6789/0 conn(0x7f417806daf0 legacy=0x7f417806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41600039c0 0x7f4160003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780b6ea0 0x7f4160004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160004d50 0x7f4160005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f417809cf70 con 0x7f41600039c0 2024-04-25T11:21:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f416000b9d0 con 0x7f4160004d50 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f416000bb80 con 0x7f41780b6ea0 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417e3ca640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41600039c0 0x7f4160003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417dbc9640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780b6ea0 0x7f4160004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417e3ca640 1 -- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780b6ea0 msgr2=0x7f4160004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417e3ca640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780b6ea0 0x7f4160004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417ebcb640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160004d50 0x7f4160005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417e3ca640 1 -- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160004d50 msgr2=0x7f4160005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417e3ca640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160004d50 0x7f4160005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.715+0000 7f417e3ca640 1 -- 172.21.15.106:0/3329953284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4160003530 con 0x7f41600039c0 2024-04-25T11:21:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f417e3ca640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41600039c0 0x7f4160003e30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f41700011e0 tx=0x7f417001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601a503b3eab0ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170007d60 con 0x7f41600039c0 2024-04-25T11:21:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41700a9020 con 0x7f41600039c0 2024-04-25T11:21:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f417d3c8640 1 -- 172.21.15.106:0/3329953284 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170007310 con 0x7f41600039c0 2024-04-25T11:21:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f418062e640 1 -- 172.21.15.106:0/3329953284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41600039c0 msgr2=0x7f4160003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f418062e640 1 --2- 172.21.15.106:0/3329953284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41600039c0 0x7f4160003e30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f41700011e0 tx=0x7f417001cba0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f417ebcb640 1 -- 172.21.15.106:0/3329953284 reap_dead start 2024-04-25T11:21:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f418062e640 1 -- 172.21.15.106:0/3329953284 shutdown_connections 2024-04-25T11:21:01.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f418062e640 1 -- 172.21.15.106:0/3329953284 >> 172.21.15.106:0/3329953284 conn(0x7f41780d00c0 msgr2=0x7f41780d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:01.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f418062e640 1 -- 172.21.15.106:0/3329953284 shutdown_connections 2024-04-25T11:21:01.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f418062e640 1 -- 172.21.15.106:0/3329953284 wait complete. 2024-04-25T11:21:01.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f418062e640 1 Processor -- start 2024-04-25T11:21:01.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.719+0000 7f418062e640 1 -- start start 2024-04-25T11:21:01.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41600039c0 0x7f417815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41780a0a10 0x7f41781619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780a0e20 0x7f4178161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f417806ea30 con 0x7f41780a0a10 2024-04-25T11:21:01.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41780d2c60 con 0x7f41600039c0 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f417816ab40 con 0x7f41780a0e20 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41780a0a10 0x7f41781619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41780a0a10 0x7f41781619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49836/0 (socket says 172.21.15.106:49836) 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 -- 172.21.15.106:0/3687549724 learned_addr learned my addr 172.21.15.106:0/3687549724 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417e3ca640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41600039c0 0x7f417815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417ebcb640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780a0e20 0x7f4178161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 -- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780a0e20 msgr2=0x7f4178161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780a0e20 0x7f4178161ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 -- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41600039c0 msgr2=0x7f417815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41600039c0 0x7f417815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 -- 172.21.15.106:0/3687549724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41780a04d0 con 0x7f41780a0a10 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417e3ca640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41600039c0 0x7f417815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f417dbc9640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41780a0a10 0x7f41781619b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4174010820 tx=0x7f4174010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:01.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f416effd640 1 -- 172.21.15.106:0/3687549724 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417400f040 con 0x7f41780a0a10 2024-04-25T11:21:01.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f416effd640 1 -- 172.21.15.106:0/3687549724 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417400eb60 con 0x7f41780a0a10 2024-04-25T11:21:01.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417809d5b0 con 0x7f41780a0a10 2024-04-25T11:21:01.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f416effd640 1 -- 172.21.15.106:0/3687549724 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417401c400 con 0x7f41780a0a10 2024-04-25T11:21:01.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4178072160 con 0x7f41780a0a10 2024-04-25T11:21:01.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41780a1610 con 0x7f41780a0a10 2024-04-25T11:21:01.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.723+0000 7f416effd640 1 -- 172.21.15.106:0/3687549724 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f417401c5a0 con 0x7f41780a0a10 2024-04-25T11:21:01.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.727+0000 7f416effd640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f416406f400 0x7f4164071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:01.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.727+0000 7f417e3ca640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f416406f400 0x7f4164071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:01.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.727+0000 7f416effd640 1 -- 172.21.15.106:0/3687549724 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f417408f850 con 0x7f41780a0a10 2024-04-25T11:21:01.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.727+0000 7f417e3ca640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f416406f400 0x7f4164071910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41700aa000 tx=0x7f41700b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:01.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.731+0000 7f416effd640 1 -- 172.21.15.106:0/3687549724 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4174018030 con 0x7f41780a0a10 2024-04-25T11:21:01.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:01.879+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f41781660f0 con 0x7f41780a0a10 2024-04-25T11:21:02.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.495+0000 7f416effd640 1 -- 172.21.15.106:0/3687549724 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f417405c800 con 0x7f41780a0a10 2024-04-25T11:21:02.493 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:21:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f416406f400 msgr2=0x7f4164071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f416406f400 0x7f4164071910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41700aa000 tx=0x7f41700b0040 comp rx=0 tx=0).stop 2024-04-25T11:21:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41780a0a10 msgr2=0x7f41781619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41780a0a10 0x7f41781619b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4174010820 tx=0x7f4174010ce0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 shutdown_connections 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41780a0e20 0x7f4178161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f416406f400 0x7f4164071910 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41700aa000 tx=0x7f41700b0040 comp rx=0 tx=0).stop 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41780a0a10 0x7f41781619b0 secure :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4174010820 tx=0x7f4174010ce0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 --2- 172.21.15.106:0/3687549724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41600039c0 0x7f417815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 >> 172.21.15.106:0/3687549724 conn(0x7f41780d00c0 msgr2=0x7f4178169310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 shutdown_connections 2024-04-25T11:21:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.503+0000 7f418062e640 1 -- 172.21.15.106:0/3687549724 wait complete. 2024-04-25T11:21:02.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:21:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff6a5e6d640 1 Processor -- start 2024-04-25T11:21:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff6a5e6d640 1 -- start start 2024-04-25T11:21:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff6a5e6d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a009ca70 0x7ff6a009ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff6a5e6d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009c380 con 0x7ff6a009bf70 2024-04-25T11:21:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff6a5e6d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a00cf560 con 0x7ff6a00cf150 2024-04-25T11:21:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff6a5e6d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a00cf830 con 0x7ff6a009ca70 2024-04-25T11:21:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff6a009bf70 0x7ff6a00b6d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45012/0 (socket says 172.21.15.106:45012) 2024-04-25T11:21:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69effd640 1 -- 172.21.15.106:0/1810694094 learned_addr learned my addr 172.21.15.106:0/1810694094 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:02.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2680939001 0 0) 0x7ff694001120 con 0x7ff6a009bf70 2024-04-25T11:21:02.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff688003530 con 0x7ff6a009bf70 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4038254733 0 0) 0x7ff688003530 con 0x7ff6a009bf70 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a009ca70 msgr2=0x7ff6a009ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a009ca70 0x7ff6a009ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 >> v1:172.21.15.134:6789/0 conn(0x7ff6a00cf150 legacy=0x7ff6a00b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a00cfb10 con 0x7ff6a009bf70 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7ff694001850 con 0x7ff6a009bf70 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 >> v1:172.21.15.106:6789/0 conn(0x7ff6a009bf70 legacy=0x7ff6a00b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff688003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a009ca70 0x7ff688004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff688004e70 0x7ff6880052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a00cf830 con 0x7ff6880039f0 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff68800bd10 con 0x7ff688004e70 2024-04-25T11:21:02.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff68800bf10 con 0x7ff6a009ca70 2024-04-25T11:21:02.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.695+0000 7ff69f7fe640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff688003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:02.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69f7fe640 1 -- 172.21.15.106:0/1810694094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a009ca70 msgr2=0x7ff688004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:02.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69f7fe640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a009ca70 0x7ff688004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69f7fe640 1 -- 172.21.15.106:0/1810694094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff688004e70 msgr2=0x7ff6880052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:02.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69f7fe640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff688004e70 0x7ff6880052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69f7fe640 1 -- 172.21.15.106:0/1810694094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff688003530 con 0x7ff6880039f0 2024-04-25T11:21:02.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69f7fe640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff688003e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff690009860 tx=0x7ff69001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8903c38d082a05d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:02.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900b0070 con 0x7ff6880039f0 2024-04-25T11:21:02.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6900ab620 con 0x7ff6880039f0 2024-04-25T11:21:02.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69e7fc640 1 -- 172.21.15.106:0/1810694094 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690009460 con 0x7ff6880039f0 2024-04-25T11:21:02.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/1810694094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 msgr2=0x7ff688003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:02.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff6a5e6d640 1 --2- 172.21.15.106:0/1810694094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff688003e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff690009860 tx=0x7ff69001c380 comp rx=0 tx=0).stop 2024-04-25T11:21:02.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff69ffff640 1 -- 172.21.15.106:0/1810694094 reap_dead start 2024-04-25T11:21:02.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/1810694094 shutdown_connections 2024-04-25T11:21:02.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/1810694094 >> 172.21.15.106:0/1810694094 conn(0x7ff6a00b5120 msgr2=0x7ff6a00b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:02.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/1810694094 shutdown_connections 2024-04-25T11:21:02.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.699+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/1810694094 wait complete. 2024-04-25T11:21:02.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 Processor -- start 2024-04-25T11:21:02.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 -- start start 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff6a015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6a01f9f70 0x7ff6a01fa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a015fa80 0x7ff6a015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a00b7f50 con 0x7ff6880039f0 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0161720 con 0x7ff6a01f9f70 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a009ce80 con 0x7ff6a015fa80 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff6a015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff6a015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49850/0 (socket says 172.21.15.106:49850) 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 -- 172.21.15.106:0/3779317504 learned_addr learned my addr 172.21.15.106:0/3779317504 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:02.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 -- 172.21.15.106:0/3779317504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a015fa80 msgr2=0x7ff6a015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69ffff640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a015fa80 0x7ff6a015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69effd640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6a01f9f70 0x7ff6a01fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a015fa80 0x7ff6a015fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 -- 172.21.15.106:0/3779317504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6a01f9f70 msgr2=0x7ff6a01fa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6a01f9f70 0x7ff6a01fa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 -- 172.21.15.106:0/3779317504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a01603d0 con 0x7ff6880039f0 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff69f7fe640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff6a015df80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff69001c1b0 tx=0x7ff688005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff687fff640 1 -- 172.21.15.106:0/3779317504 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900b0040 con 0x7ff6880039f0 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff687fff640 1 -- 172.21.15.106:0/3779317504 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6900bd070 con 0x7ff6880039f0 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff687fff640 1 -- 172.21.15.106:0/3779317504 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900ab9d0 con 0x7ff6880039f0 2024-04-25T11:21:02.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a00cd1d0 con 0x7ff6880039f0 2024-04-25T11:21:02.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.703+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a00cc410 con 0x7ff6880039f0 2024-04-25T11:21:02.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.707+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a0050830 con 0x7ff6880039f0 2024-04-25T11:21:02.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.707+0000 7ff687fff640 1 -- 172.21.15.106:0/3779317504 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff6900b3070 con 0x7ff6880039f0 2024-04-25T11:21:02.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.707+0000 7ff687fff640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff68c06f430 0x7ff68c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:02.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.707+0000 7ff687fff640 1 -- 172.21.15.106:0/3779317504 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7ff6901292f0 con 0x7ff6880039f0 2024-04-25T11:21:02.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.707+0000 7ff69effd640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff68c06f430 0x7ff68c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:02.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.707+0000 7ff69effd640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff68c06f430 0x7ff68c071940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00d0e90 tx=0x7ff694009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.711+0000 7ff687fff640 1 -- 172.21.15.106:0/3779317504 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff6a01603d0 con 0x7ff6880039f0 2024-04-25T11:21:02.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:02.875+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6a00b4e20 con 0x7ff6880039f0 2024-04-25T11:21:03.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.507+0000 7ff687fff640 1 -- 172.21.15.106:0/3779317504 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7ff690016090 con 0x7ff6880039f0 2024-04-25T11:21:03.506 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-25T11:21:03.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff68c06f430 msgr2=0x7ff68c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff68c06f430 0x7ff68c071940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00d0e90 tx=0x7ff694009040 comp rx=0 tx=0).stop 2024-04-25T11:21:03.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 msgr2=0x7ff6a015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff6a015df80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff69001c1b0 tx=0x7ff688005840 comp rx=0 tx=0).stop 2024-04-25T11:21:03.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 shutdown_connections 2024-04-25T11:21:03.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6a015fa80 0x7ff6a015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6a01f9f70 0x7ff6a01fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff68c06f430 0x7ff68c071940 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 --2- 172.21.15.106:0/3779317504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6880039f0 0x7ff6a015df80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 >> 172.21.15.106:0/3779317504 conn(0x7ff6a00b5120 msgr2=0x7ff6a00d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:03.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 shutdown_connections 2024-04-25T11:21:03.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.511+0000 7ff6a5e6d640 1 -- 172.21.15.106:0/3779317504 wait complete. 2024-04-25T11:21:03.536 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:21:03.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e9cc6640 1 Processor -- start 2024-04-25T11:21:03.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e9cc6640 1 -- start start 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e9cc6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3e40d0cb0 0x7fb3e4167c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e9cc6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e406ec70 con 0x7fb3e40d1e50 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e9cc6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e406ef50 con 0x7fb3e40cf4d0 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e9cc6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e40707b0 con 0x7fb3e40d0cb0 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3dbfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb3e40d1e50 0x7fb3e41644d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45026/0 (socket says 172.21.15.106:45026) 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3dbfff640 1 -- 172.21.15.106:0/3279101324 learned_addr learned my addr 172.21.15.106:0/3279101324 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e3fff640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3e40d0cb0 0x7fb3e4167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197647383 0 0) 0x7fb3e406ec70 con 0x7fb3e40d1e50 2024-04-25T11:21:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3d0003530 con 0x7fb3e40d1e50 2024-04-25T11:21:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2865257937 0 0) 0x7fb3d0003530 con 0x7fb3e40d1e50 2024-04-25T11:21:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3e40d0cb0 msgr2=0x7fb3e4167c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3e40d0cb0 0x7fb3e4167c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 >> v1:172.21.15.134:6789/0 conn(0x7fb3e40cf4d0 legacy=0x7fb3e40d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e406f230 con 0x7fb3e40d1e50 2024-04-25T11:21:03.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fb3cc0016a0 con 0x7fb3e40d1e50 2024-04-25T11:21:03.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 >> v1:172.21.15.106:6789/0 conn(0x7fb3e40d1e50 legacy=0x7fb3e41644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3d00039f0 0x7fb3d0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:03.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d0004810 0x7fb3d0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:03.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3e40d0cb0 0x7fb3d00052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:03.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d000bb60 con 0x7fb3d00039f0 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d000bd10 con 0x7fb3d0004810 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d000bf10 con 0x7fb3e40d0cb0 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e37fe640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3d00039f0 0x7fb3d0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3dbfff640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d0004810 0x7fb3d0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e3fff640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3e40d0cb0 0x7fb3d00052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e37fe640 1 -- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3e40d0cb0 msgr2=0x7fb3d00052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e37fe640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3e40d0cb0 0x7fb3d00052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e37fe640 1 -- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d0004810 msgr2=0x7fb3d0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e37fe640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d0004810 0x7fb3d0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.691+0000 7fb3e37fe640 1 -- 172.21.15.106:0/3279101324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d0003530 con 0x7fb3d00039f0 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e37fe640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3d00039f0 0x7fb3d0003e60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4001750 tx=0x7fb3d401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69eb0090df3a11ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40013a0 con 0x7fb3d00039f0 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d40052f0 con 0x7fb3d00039f0 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e2ffd640 1 -- 172.21.15.106:0/3279101324 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4003840 con 0x7fb3d00039f0 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3279101324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3d00039f0 msgr2=0x7fb3d0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e9cc6640 1 --2- 172.21.15.106:0/3279101324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3d00039f0 0x7fb3d0003e60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4001750 tx=0x7fb3d401c3e0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e3fff640 1 -- 172.21.15.106:0/3279101324 reap_dead start 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3279101324 shutdown_connections 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3279101324 >> 172.21.15.106:0/3279101324 conn(0x7fb3e406c9f0 msgr2=0x7fb3e406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3279101324 shutdown_connections 2024-04-25T11:21:03.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.695+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3279101324 wait complete. 2024-04-25T11:21:03.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 Processor -- start 2024-04-25T11:21:03.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 -- start start 2024-04-25T11:21:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3d00039f0 0x7fb3e415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4164580 0x7fb3e4153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3e41540e0 0x7fb3e41fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e406ea20 con 0x7fb3e41540e0 2024-04-25T11:21:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4168620 con 0x7fb3e4164580 2024-04-25T11:21:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e41687a0 con 0x7fb3d00039f0 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3e41540e0 0x7fb3e41fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3e41540e0 0x7fb3e41fc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49866/0 (socket says 172.21.15.106:49866) 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 -- 172.21.15.106:0/3025286391 learned_addr learned my addr 172.21.15.106:0/3025286391 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3dbfff640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4164580 0x7fb3e4153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e37fe640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3d00039f0 0x7fb3e415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 -- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3d00039f0 msgr2=0x7fb3e415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3d00039f0 0x7fb3e415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 -- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4164580 msgr2=0x7fb3e4153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4164580 0x7fb3e4153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 -- 172.21.15.106:0/3025286391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e41544f0 con 0x7fb3e41540e0 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e37fe640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3d00039f0 0x7fb3e415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3dbfff640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4164580 0x7fb3e4153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e3fff640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3e41540e0 0x7fb3e41fc5a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40d2400 tx=0x7fb3dc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e0ff9640 1 -- 172.21.15.106:0/3025286391 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc01c070 con 0x7fb3e41540e0 2024-04-25T11:21:03.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e0ff9640 1 -- 172.21.15.106:0/3025286391 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc017400 con 0x7fb3e41540e0 2024-04-25T11:21:03.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e0ff9640 1 -- 172.21.15.106:0/3025286391 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0177f0 con 0x7fb3e41540e0 2024-04-25T11:21:03.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e415c2e0 con 0x7fb3e41540e0 2024-04-25T11:21:03.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.699+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e4165320 con 0x7fb3e41540e0 2024-04-25T11:21:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.703+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e4164db0 con 0x7fb3e41540e0 2024-04-25T11:21:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.703+0000 7fb3e0ff9640 1 -- 172.21.15.106:0/3025286391 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb3dc01f070 con 0x7fb3e41540e0 2024-04-25T11:21:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.703+0000 7fb3e0ff9640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb3c406f400 0x7fb3c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.703+0000 7fb3e0ff9640 1 -- 172.21.15.106:0/3025286391 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fb3dc08f0d0 con 0x7fb3e41540e0 2024-04-25T11:21:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.703+0000 7fb3e37fe640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb3c406f400 0x7fb3c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.703+0000 7fb3e37fe640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb3c406f400 0x7fb3c4071910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4072820 tx=0x7fb3d40011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:03.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.707+0000 7fb3e0ff9640 1 -- 172.21.15.106:0/3025286391 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb3e41544f0 con 0x7fb3e41540e0 2024-04-25T11:21:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.870+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb3e4050750 con 0x7fb3e41540e0 2024-04-25T11:21:03.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.874+0000 7fb3e0ff9640 1 -- 172.21.15.106:0/3025286391 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fb3dc012070 con 0x7fb3e41540e0 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb3c406f400 msgr2=0x7fb3c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb3c406f400 0x7fb3c4071910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4072820 tx=0x7fb3d40011a0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3e41540e0 msgr2=0x7fb3e41fc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3e41540e0 0x7fb3e41fc5a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40d2400 tx=0x7fb3dc00f4a0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 shutdown_connections 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fb3c406f400 0x7fb3c4071910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3e41540e0 0x7fb3e41fc5a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4164580 0x7fb3e4153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 --2- 172.21.15.106:0/3025286391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3d00039f0 0x7fb3e415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:03.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 >> 172.21.15.106:0/3025286391 conn(0x7fb3e406c9f0 msgr2=0x7fb3e4072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:03.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 shutdown_connections 2024-04-25T11:21:03.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:03.878+0000 7fb3e9cc6640 1 -- 172.21.15.106:0/3025286391 wait complete. 2024-04-25T11:21:03.897 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_readahead.TestReadahead.test_flush' 2024-04-25T11:21:04.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f688478d640 1 Processor -- start 2024-04-25T11:21:04.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f688478d640 1 -- start start 2024-04-25T11:21:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f688478d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f687c15e0a0 0x7f687c073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f688478d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f687c0ce850 0x7f687c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f688478d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c0cf1a0 con 0x7f687c06f5f0 2024-04-25T11:21:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f688478d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f687c0d1cf0 con 0x7f687c0ce850 2024-04-25T11:21:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f688478d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f687c0ce050 con 0x7f687c15e0a0 2024-04-25T11:21:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6882d2a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f687c06f5f0 0x7f687c06fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45028/0 (socket says 172.21.15.106:45028) 2024-04-25T11:21:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6882d2a640 1 -- 172.21.15.106:0/2522331399 learned_addr learned my addr 172.21.15.106:0/2522331399 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6882529640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f687c15e0a0 0x7f687c073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881d28640 1 --2- 172.21.15.106:0/2522331399 >> v2:172.21.15.134:3300/0 conn(0x7f687c0ce850 0x7f687c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839943110 0 0) 0x7f68740014a0 con 0x7f687c06f5f0 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6868003530 con 0x7f687c06f5f0 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4119808284 0 0) 0x7f6868003530 con 0x7f687c06f5f0 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f687c15e0a0 msgr2=0x7f687c073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f687c15e0a0 0x7f687c073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 >> v2:172.21.15.134:3300/0 conn(0x7f687c0ce850 msgr2=0x7f687c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 --2- 172.21.15.106:0/2522331399 >> v2:172.21.15.134:3300/0 conn(0x7f687c0ce850 0x7f687c0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c15e6b0 con 0x7f687c06f5f0 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f6874001bd0 con 0x7f687c06f5f0 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 >> v1:172.21.15.106:6789/0 conn(0x7f687c06f5f0 legacy=0x7f687c06fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68680039c0 0x7f6868003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6868004810 0x7f6868004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f687c15e0a0 0x7f6868005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f687c0ce050 con 0x7f6868004810 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f686800b9d0 con 0x7f68680039c0 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f686800bb50 con 0x7f687c15e0a0 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881d28640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6868004810 0x7f6868004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6882529640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68680039c0 0x7f6868003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881d28640 1 -- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f687c15e0a0 msgr2=0x7f6868005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:04.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881d28640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f687c15e0a0 0x7f6868005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881d28640 1 -- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68680039c0 msgr2=0x7f6868003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:04.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881d28640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68680039c0 0x7f6868003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.046+0000 7f6881d28640 1 -- 172.21.15.106:0/2522331399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6868003530 con 0x7f6868004810 2024-04-25T11:21:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f6881d28640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6868004810 0x7f6868004c20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f686c009ae0 tx=0x7f686c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39eb3b581d5e2fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0b2070 con 0x7f6868004810 2024-04-25T11:21:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c0a55e0 con 0x7f6868004810 2024-04-25T11:21:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f6881527640 1 -- 172.21.15.106:0/2522331399 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0096b0 con 0x7f6868004810 2024-04-25T11:21:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f688478d640 1 -- 172.21.15.106:0/2522331399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6868004810 msgr2=0x7f6868004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f688478d640 1 --2- 172.21.15.106:0/2522331399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6868004810 0x7f6868004c20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f686c009ae0 tx=0x7f686c01c9f0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f6882d2a640 1 -- 172.21.15.106:0/2522331399 reap_dead start 2024-04-25T11:21:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f688478d640 1 -- 172.21.15.106:0/2522331399 shutdown_connections 2024-04-25T11:21:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f688478d640 1 -- 172.21.15.106:0/2522331399 >> 172.21.15.106:0/2522331399 conn(0x7f687c06c7c0 msgr2=0x7f687c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f688478d640 1 -- 172.21.15.106:0/2522331399 shutdown_connections 2024-04-25T11:21:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.050+0000 7f688478d640 1 -- 172.21.15.106:0/2522331399 wait complete. 2024-04-25T11:21:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 Processor -- start 2024-04-25T11:21:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 -- start start 2024-04-25T11:21:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6868004810 0x7f687c12e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f687c1315f0 0x7f687c130240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f687c1f38e0 0x7f687c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f687c15ef50 con 0x7f687c1315f0 2024-04-25T11:21:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f687c15ec70 con 0x7f687c1f38e0 2024-04-25T11:21:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f687c0d06e0 con 0x7f6868004810 2024-04-25T11:21:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f687c1315f0 0x7f687c130240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f687c1315f0 0x7f687c130240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49882/0 (socket says 172.21.15.106:49882) 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 -- 172.21.15.106:0/2865513622 learned_addr learned my addr 172.21.15.106:0/2865513622 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6882d2a640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f687c1f38e0 0x7f687c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6882529640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6868004810 0x7f687c12e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 -- 172.21.15.106:0/2865513622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6868004810 msgr2=0x7f687c12e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6868004810 0x7f687c12e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 -- 172.21.15.106:0/2865513622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f687c1f38e0 msgr2=0x7f687c0c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f687c1f38e0 0x7f687c0c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 -- 172.21.15.106:0/2865513622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c12f1a0 con 0x7f687c1315f0 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6881d28640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f687c1315f0 0x7f687c130240 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f686c009ae0 tx=0x7f6868005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6872ffd640 1 -- 172.21.15.106:0/2865513622 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0b2040 con 0x7f687c1315f0 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6872ffd640 1 -- 172.21.15.106:0/2865513622 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c0a5e10 con 0x7f687c1315f0 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c12f480 con 0x7f687c1315f0 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f6872ffd640 1 -- 172.21.15.106:0/2865513622 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c004020 con 0x7f687c1315f0 2024-04-25T11:21:04.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.054+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c12fa40 con 0x7f687c1315f0 2024-04-25T11:21:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.058+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c12eec0 con 0x7f687c1315f0 2024-04-25T11:21:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.058+0000 7f6872ffd640 1 -- 172.21.15.106:0/2865513622 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f686c0c0020 con 0x7f687c1315f0 2024-04-25T11:21:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.058+0000 7f6872ffd640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f686006f400 0x7f6860071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.058+0000 7f6872ffd640 1 -- 172.21.15.106:0/2865513622 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f686c129000 con 0x7f687c1315f0 2024-04-25T11:21:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.058+0000 7f6882529640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f686006f400 0x7f6860071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.058+0000 7f6882529640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f686006f400 0x7f6860071910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6878009160 tx=0x7f6878008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:04.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.062+0000 7f6872ffd640 1 -- 172.21.15.106:0/2865513622 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f686c0167c0 con 0x7f687c1315f0 2024-04-25T11:21:04.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.286+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_readahead.TestReadahead.test_flush"]} v 0) -- 0x7f687c0b2e40 con 0x7f687c1315f0 2024-04-25T11:21:04.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.506+0000 7f6872ffd640 1 -- 172.21.15.106:0/2865513622 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_readahead.TestReadahead.test_flush"]}]=0 v113) ==== 131+0+0 (secure 0 0 0) 0x7f686c0f5ba0 con 0x7f687c1315f0 2024-04-25T11:21:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f686006f400 msgr2=0x7f6860071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f686006f400 0x7f6860071910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6878009160 tx=0x7f6878008040 comp rx=0 tx=0).stop 2024-04-25T11:21:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f687c1315f0 msgr2=0x7f687c130240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f687c1315f0 0x7f687c130240 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f686c009ae0 tx=0x7f6868005840 comp rx=0 tx=0).stop 2024-04-25T11:21:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 shutdown_connections 2024-04-25T11:21:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f687c1f38e0 0x7f687c0c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f686006f400 0x7f6860071910 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f687c1315f0 0x7f687c130240 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 --2- 172.21.15.106:0/2865513622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6868004810 0x7f687c12e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 >> 172.21.15.106:0/2865513622 conn(0x7f687c06c7c0 msgr2=0x7f687c0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 shutdown_connections 2024-04-25T11:21:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:04.510+0000 7f688478d640 1 -- 172.21.15.106:0/2865513622 wait complete. 2024-04-25T11:21:04.534 INFO:tasks.cephfs_test_runner:test_flush (tasks.cephfs.test_readahead.TestReadahead) ... ok 2024-04-25T11:21:04.535 INFO:tasks.cephfs_test_runner: 2024-04-25T11:21:04.535 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T11:21:04.535 INFO:tasks.cephfs_test_runner:Ran 1 test in 64.747s 2024-04-25T11:21:04.535 INFO:tasks.cephfs_test_runner: 2024-04-25T11:21:04.535 INFO:tasks.cephfs_test_runner:OK 2024-04-25T11:21:04.536 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T11:21:04.546 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T11:21:04.590 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T11:21:04.590 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:21:04.590 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:04.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16979 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:21:04.596 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T11:21:04.596 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:04.647 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16979 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:21:04.647 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:21:04.647 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:04.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336524k,nr_inodes=4084131,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274692k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16940 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274688k,nr_inodes=818672,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274692k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:21:04.654 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T11:21:04.654 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:04.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336524k,nr_inodes=4084131,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274692k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16940 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274688k,nr_inodes=818672,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274692k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:21:04.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:04.700 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T11:21:04.706 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:21:04.706 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:21:04.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:04.706 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:21:04.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:04.762 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:21:04.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:04.849 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-25T11:21:05.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:05.015 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-04-25T11:21:05.021 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:21:05.021 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:21:05.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:05.022 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:21:05.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:05.077 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:21:05.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:05.144 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-04-25T11:21:05.308 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T11:21:05.318 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:21:05.318 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:21:05.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8af0050640 1 Processor -- start 2024-04-25T11:21:05.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8af0050640 1 -- start start 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8af0050640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae815e0a0 0x7f8ae8073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8af0050640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8ae80d0ce0 0x7f8ae80d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8af0050640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae80ce050 con 0x7f8ae80ce850 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8af0050640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae815e6b0 con 0x7f8ae80d0ce0 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8af0050640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae815e990 con 0x7f8ae815e0a0 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aee5ed640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8ae80d0ce0 0x7f8ae80d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aed5eb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8ae80ce850 0x7f8ae80cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45862/0 (socket says 172.21.15.106:45862) 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aed5eb640 1 -- 172.21.15.106:0/1437760900 learned_addr learned my addr 172.21.15.106:0/1437760900 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aee5ed640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8ae80d0ce0 0x7f8ae80d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35978/0 (socket says 172.21.15.106:35978) 2024-04-25T11:21:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aeddec640 1 --2- 172.21.15.106:0/1437760900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae815e0a0 0x7f8ae8073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aee5ed640 1 -- 172.21.15.106:0/1437760900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae815e0a0 msgr2=0x7f8ae8073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aee5ed640 1 --2- 172.21.15.106:0/1437760900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae815e0a0 0x7f8ae8073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aee5ed640 1 -- 172.21.15.106:0/1437760900 >> v1:172.21.15.106:6789/0 conn(0x7f8ae80ce850 legacy=0x7f8ae80cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aecdea640 1 -- 172.21.15.106:0/1437760900 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2041873939 0 0) 0x7f8ae80ce050 con 0x7f8ae80ce850 2024-04-25T11:21:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.486+0000 7f8aee5ed640 1 -- 172.21.15.106:0/1437760900 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae815ec70 con 0x7f8ae80d0ce0 2024-04-25T11:21:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8aee5ed640 1 --2- 172.21.15.106:0/1437760900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae80d0ce0 0x7f8ae80d10f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80097c0 tx=0x7f8ad801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4756ca6588a9c19d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8aecdea640 1 -- 172.21.15.106:0/1437760900 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80b0070 con 0x7f8ae80d0ce0 2024-04-25T11:21:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8aecdea640 1 -- 172.21.15.106:0/1437760900 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad80ab600 con 0x7f8ae80d0ce0 2024-04-25T11:21:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8aecdea640 1 -- 172.21.15.106:0/1437760900 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80ab9d0 con 0x7f8ae80d0ce0 2024-04-25T11:21:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8af0050640 1 -- 172.21.15.106:0/1437760900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae80d0ce0 msgr2=0x7f8ae80d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8af0050640 1 --2- 172.21.15.106:0/1437760900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae80d0ce0 0x7f8ae80d10f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80097c0 tx=0x7f8ad801c460 comp rx=0 tx=0).stop 2024-04-25T11:21:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8af0050640 1 -- 172.21.15.106:0/1437760900 shutdown_connections 2024-04-25T11:21:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8af0050640 1 --2- 172.21.15.106:0/1437760900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae80d0ce0 0x7f8ae80d10f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8af0050640 1 --2- 172.21.15.106:0/1437760900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae815e0a0 0x7f8ae8073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8af0050640 1 -- 172.21.15.106:0/1437760900 >> 172.21.15.106:0/1437760900 conn(0x7f8ae806c7c0 msgr2=0x7f8ae806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8af0050640 1 -- 172.21.15.106:0/1437760900 shutdown_connections 2024-04-25T11:21:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.490+0000 7f8af0050640 1 -- 172.21.15.106:0/1437760900 wait complete. 2024-04-25T11:21:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 Processor -- start 2024-04-25T11:21:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 -- start start 2024-04-25T11:21:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ae80ce850 0x7f8ae81462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae80d0ce0 0x7f8ae8146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae815e0a0 0x7f8ae813a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae816a870 con 0x7f8ae815e0a0 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae816a6f0 con 0x7f8ae80ce850 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae816a9f0 con 0x7f8ae80d0ce0 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae815e0a0 0x7f8ae813a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae815e0a0 0x7f8ae813a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35988/0 (socket says 172.21.15.106:35988) 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 -- 172.21.15.106:0/3329220320 learned_addr learned my addr 172.21.15.106:0/3329220320 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aeddec640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ae80ce850 0x7f8ae81462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aed5eb640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae80d0ce0 0x7f8ae8146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 -- 172.21.15.106:0/3329220320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae80d0ce0 msgr2=0x7f8ae8146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae80d0ce0 0x7f8ae8146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 -- 172.21.15.106:0/3329220320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ae80ce850 msgr2=0x7f8ae81462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ae80ce850 0x7f8ae81462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 -- 172.21.15.106:0/3329220320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae8147050 con 0x7f8ae815e0a0 2024-04-25T11:21:05.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8aee5ed640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae815e0a0 0x7f8ae813a560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80097c0 tx=0x7f8ad801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8ae67fc640 1 -- 172.21.15.106:0/3329220320 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80bc070 con 0x7f8ae815e0a0 2024-04-25T11:21:05.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8ae67fc640 1 -- 172.21.15.106:0/3329220320 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad80b1040 con 0x7f8ae815e0a0 2024-04-25T11:21:05.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae813c230 con 0x7f8ae815e0a0 2024-04-25T11:21:05.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.494+0000 7f8ae67fc640 1 -- 172.21.15.106:0/3329220320 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80b0050 con 0x7f8ae815e0a0 2024-04-25T11:21:05.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.498+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae813c7f0 con 0x7f8ae815e0a0 2024-04-25T11:21:05.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.498+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae8147280 con 0x7f8ae815e0a0 2024-04-25T11:21:05.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.498+0000 7f8ae67fc640 1 -- 172.21.15.106:0/3329220320 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8ad80bf070 con 0x7f8ae815e0a0 2024-04-25T11:21:05.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.498+0000 7f8ae67fc640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8ad406f400 0x7f8ad4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.498+0000 7f8ae67fc640 1 -- 172.21.15.106:0/3329220320 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f8ad81288d0 con 0x7f8ae815e0a0 2024-04-25T11:21:05.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.498+0000 7f8aeddec640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8ad406f400 0x7f8ad4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.498+0000 7f8aeddec640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8ad406f400 0x7f8ad4071910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000b930 tx=0x7f8ae0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.502+0000 7f8ae67fc640 1 -- 172.21.15.106:0/3329220320 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ad8016050 con 0x7f8ae815e0a0 2024-04-25T11:21:05.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.658+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ae8143490 con 0x7f8ad406f400 2024-04-25T11:21:05.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8ae67fc640 1 -- 172.21.15.106:0/3329220320 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13404 (secure 0 0 0) 0x7f8ae8143490 con 0x7f8ad406f400 2024-04-25T11:21:05.662 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:21:05.662 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:21:05.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8ad406f400 msgr2=0x7f8ad4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8ad406f400 0x7f8ad4071910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000b930 tx=0x7f8ae0008040 comp rx=0 tx=0).stop 2024-04-25T11:21:05.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae815e0a0 msgr2=0x7f8ae813a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae815e0a0 0x7f8ae813a560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80097c0 tx=0x7f8ad801aca0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 shutdown_connections 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8ad406f400 0x7f8ad4071910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ae815e0a0 0x7f8ae813a560 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ae80d0ce0 0x7f8ae8146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 --2- 172.21.15.106:0/3329220320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ae80ce850 0x7f8ae81462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 >> 172.21.15.106:0/3329220320 conn(0x7f8ae806c7c0 msgr2=0x7f8ae8142390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 shutdown_connections 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.662+0000 7f8af0050640 1 -- 172.21.15.106:0/3329220320 wait complete. 2024-04-25T11:21:05.682 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-25T11:21:04.123156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":285824,"kb_used_data":69696,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754688896,"statfs":{"total":773094113280,"available":772801429504,"internally_reserved":0,"allocated":71368704,"data_stored":69494992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33570555,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-5,"num_read_kb":-512,"num_write":-68,"num_write_kb":-32802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001496"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":134,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:34.878303+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:20:34.878303+0000","last_peered":"2024-04-25T11:20:34.878303+0000","last_clean":"2024-04-25T11:20:34.878303+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:20:34.878303+0000","last_undegraded":"2024-04-25T11:20:34.878303+0000","last_fullsized":"2024-04-25T11:20:34.878303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":5044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":5165056,"data_stored":4931370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":4575232,"data_stored":4341528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43772,"kb_used_data":16756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328068,"statfs":{"total":96636764160,"available":96591941632,"internally_reserved":0,"allocated":17158144,"data_stored":16924496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43772,"kb_used_data":16756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328068,"statfs":{"total":96636764160,"available":96591941632,"internally_reserved":0,"allocated":17158144,"data_stored":16923928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":8584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":8790016,"data_stored":8550704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":4464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":4571136,"data_stored":4340982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":8584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":8790016,"data_stored":8550704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":5040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":5160960,"data_stored":4931280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:21:05.683 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:21:05.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7902fa7640 1 Processor -- start 2024-04-25T11:21:05.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7902fa7640 1 -- start start 2024-04-25T11:21:05.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7902fa7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f78fc15fda0 0x7f78fc0b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7902fa7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78fc0d5700 0x7f78fc0d5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7902fa7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78fc0d6050 con 0x7f78fc0d4480 2024-04-25T11:21:05.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7902fa7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f78fc1601b0 con 0x7f78fc0d5700 2024-04-25T11:21:05.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7902fa7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f78fc0b91c0 con 0x7f78fc15fda0 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f78fbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78fc0d5700 0x7f78fc0d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f78fbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78fc0d5700 0x7f78fc0d5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36000/0 (socket says 172.21.15.106:36000) 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f78fbfff640 1 -- 172.21.15.106:0/1541991571 learned_addr learned my addr 172.21.15.106:0/1541991571 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7901544640 1 --1- 172.21.15.106:0/1541991571 >> v1:172.21.15.106:6789/0 conn(0x7f78fc0d4480 0x7f78fc0d4890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45866/0 (socket says 172.21.15.106:45866) 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f7900d43640 1 --2- 172.21.15.106:0/1541991571 >> v2:172.21.15.134:3300/0 conn(0x7f78fc15fda0 0x7f78fc0b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f78fbfff640 1 -- 172.21.15.106:0/1541991571 >> v2:172.21.15.134:3300/0 conn(0x7f78fc15fda0 msgr2=0x7f78fc0b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f78fbfff640 1 --2- 172.21.15.106:0/1541991571 >> v2:172.21.15.134:3300/0 conn(0x7f78fc15fda0 0x7f78fc0b8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f78fbfff640 1 -- 172.21.15.106:0/1541991571 >> v1:172.21.15.106:6789/0 conn(0x7f78fc0d4480 legacy=0x7f78fc0d4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f78fb7fe640 1 -- 172.21.15.106:0/1541991571 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2376855767 0 0) 0x7f78ec001120 con 0x7f78fc0d4480 2024-04-25T11:21:05.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.830+0000 7f78fbfff640 1 -- 172.21.15.106:0/1541991571 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc163db0 con 0x7f78fc0d5700 2024-04-25T11:21:05.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f78fbfff640 1 --2- 172.21.15.106:0/1541991571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc0d5700 0x7f78fc0d5b10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f78fc08e460 tx=0x7f78f001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c844d3751e901a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f78fb7fe640 1 -- 172.21.15.106:0/1541991571 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00a57d0 con 0x7f78fc0d5700 2024-04-25T11:21:05.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f78fb7fe640 1 -- 172.21.15.106:0/1541991571 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f00a5970 con 0x7f78fc0d5700 2024-04-25T11:21:05.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f78fb7fe640 1 -- 172.21.15.106:0/1541991571 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00a5d40 con 0x7f78fc0d5700 2024-04-25T11:21:05.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f7902fa7640 1 -- 172.21.15.106:0/1541991571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc0d5700 msgr2=0x7f78fc0d5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1541991571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc0d5700 0x7f78fc0d5b10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f78fc08e460 tx=0x7f78f001c9f0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f7902fa7640 1 -- 172.21.15.106:0/1541991571 shutdown_connections 2024-04-25T11:21:05.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1541991571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc0d5700 0x7f78fc0d5b10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1541991571 >> v2:172.21.15.134:3300/0 conn(0x7f78fc15fda0 0x7f78fc0b8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f7902fa7640 1 -- 172.21.15.106:0/1541991571 >> 172.21.15.106:0/1541991571 conn(0x7f78fc06d8e0 msgr2=0x7f78fc06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:05.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f7902fa7640 1 -- 172.21.15.106:0/1541991571 shutdown_connections 2024-04-25T11:21:05.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.834+0000 7f7902fa7640 1 -- 172.21.15.106:0/1541991571 wait complete. 2024-04-25T11:21:05.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 Processor -- start 2024-04-25T11:21:05.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 -- start start 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78fc0d4480 0x7f78fc0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78fc0d5700 0x7f78fc1f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc15fda0 0x7f78fc1f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0d5120 con 0x7f78fc15fda0 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0d4fa0 con 0x7f78fc0d4480 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f78fc0b4a20 con 0x7f78fc0d5700 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc15fda0 0x7f78fc1f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc15fda0 0x7f78fc1f9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36010/0 (socket says 172.21.15.106:36010) 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 -- 172.21.15.106:0/1155868471 learned_addr learned my addr 172.21.15.106:0/1155868471 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7900d43640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78fc0d4480 0x7f78fc0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f78fbfff640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78fc0d5700 0x7f78fc1f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 -- 172.21.15.106:0/1155868471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78fc0d5700 msgr2=0x7f78fc1f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78fc0d5700 0x7f78fc1f98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 -- 172.21.15.106:0/1155868471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78fc0d4480 msgr2=0x7f78fc0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78fc0d4480 0x7f78fc0cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 -- 172.21.15.106:0/1155868471 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc161e20 con 0x7f78fc15fda0 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7901544640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc15fda0 0x7f78fc1f9df0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c3e0 tx=0x7f78ec00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f78f97fa640 1 -- 172.21.15.106:0/1155868471 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0074e0 con 0x7f78fc15fda0 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f78f97fa640 1 -- 172.21.15.106:0/1155868471 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78ec0047f0 con 0x7f78fc15fda0 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78fc162100 con 0x7f78fc15fda0 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f78f97fa640 1 -- 172.21.15.106:0/1155868471 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec004be0 con 0x7f78fc15fda0 2024-04-25T11:21:05.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.838+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78fc1626c0 con 0x7f78fc15fda0 2024-04-25T11:21:05.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.842+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78fc161b90 con 0x7f78fc15fda0 2024-04-25T11:21:05.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.842+0000 7f78f97fa640 1 -- 172.21.15.106:0/1155868471 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f78ec011070 con 0x7f78fc15fda0 2024-04-25T11:21:05.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.842+0000 7f78f97fa640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f78e806f400 0x7f78e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.842+0000 7f78f97fa640 1 -- 172.21.15.106:0/1155868471 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f78ec08f110 con 0x7f78fc15fda0 2024-04-25T11:21:05.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.842+0000 7f7900d43640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f78e806f400 0x7f78e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.842+0000 7f7900d43640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f78e806f400 0x7f78e8071910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78f400b930 tx=0x7f78f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:05.846+0000 7f78f97fa640 1 -- 172.21.15.106:0/1155868471 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f78fc0d6050 con 0x7f78fc15fda0 2024-04-25T11:21:06.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.002+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78fc0d3550 con 0x7f78e806f400 2024-04-25T11:21:06.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f78f97fa640 1 -- 172.21.15.106:0/1155868471 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13404 (secure 0 0 0) 0x7f78fc0d3550 con 0x7f78e806f400 2024-04-25T11:21:06.006 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:21:06.006 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f78e806f400 msgr2=0x7f78e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f78e806f400 0x7f78e8071910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78f400b930 tx=0x7f78f4008040 comp rx=0 tx=0).stop 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc15fda0 msgr2=0x7f78fc1f9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc15fda0 0x7f78fc1f9df0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c3e0 tx=0x7f78ec00c8a0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 shutdown_connections 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f78e806f400 0x7f78e8071910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78fc15fda0 0x7f78fc1f9df0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78fc0d5700 0x7f78fc1f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 --2- 172.21.15.106:0/1155868471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78fc0d4480 0x7f78fc0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 >> 172.21.15.106:0/1155868471 conn(0x7f78fc06d8e0 msgr2=0x7f78fc06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:06.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 shutdown_connections 2024-04-25T11:21:06.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.006+0000 7f7902fa7640 1 -- 172.21.15.106:0/1155868471 wait complete. 2024-04-25T11:21:06.027 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-25T11:21:04.123156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":285824,"kb_used_data":69696,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754688896,"statfs":{"total":773094113280,"available":772801429504,"internally_reserved":0,"allocated":71368704,"data_stored":69494992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33570555,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-5,"num_read_kb":-512,"num_write":-68,"num_write_kb":-32802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001496"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":134,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T11:20:34.878303+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:20:34.878303+0000","last_peered":"2024-04-25T11:20:34.878303+0000","last_clean":"2024-04-25T11:20:34.878303+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:20:34.878303+0000","last_undegraded":"2024-04-25T11:20:34.878303+0000","last_fullsized":"2024-04-25T11:20:34.878303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":5044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":5165056,"data_stored":4931370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":4575232,"data_stored":4341528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43772,"kb_used_data":16756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328068,"statfs":{"total":96636764160,"available":96591941632,"internally_reserved":0,"allocated":17158144,"data_stored":16924496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43772,"kb_used_data":16756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328068,"statfs":{"total":96636764160,"available":96591941632,"internally_reserved":0,"allocated":17158144,"data_stored":16923928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":8584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":8790016,"data_stored":8550704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":4464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":4571136,"data_stored":4340982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":8584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":8790016,"data_stored":8550704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":5040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":5160960,"data_stored":4931280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:21:06.028 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:21:06.028 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:21:06.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff45592c640 1 Processor -- start 2024-04-25T11:21:06.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff45592c640 1 -- start start 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff45592c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4500d12d0 0x7ff4500d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff45592c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4500d1c20 con 0x7ff450163d40 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff45592c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4500b50a0 con 0x7ff45006fc10 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff45592c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff4501638b0 con 0x7ff4500d12d0 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff450163d40 0x7ff450164150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45868/0 (socket says 172.21.15.106:45868) 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44f7fe640 1 -- 172.21.15.106:0/241114186 learned_addr learned my addr 172.21.15.106:0/241114186 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44e7fc640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4500d12d0 0x7ff4500d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4077394662 0 0) 0x7ff448001120 con 0x7ff450163d40 2024-04-25T11:21:06.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff438003530 con 0x7ff450163d40 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2528413623 0 0) 0x7ff438003530 con 0x7ff450163d40 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4500d12d0 msgr2=0x7ff4500d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4500d12d0 0x7ff4500d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 >> v1:172.21.15.134:6789/0 conn(0x7ff45006fc10 legacy=0x7ff4500b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff450070730 con 0x7ff450163d40 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7ff448001850 con 0x7ff450163d40 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 >> v1:172.21.15.106:6789/0 conn(0x7ff450163d40 legacy=0x7ff450164150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380039f0 0x7ff438003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4500d12d0 0x7ff438004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438004d50 0x7ff438005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501638b0 con 0x7ff438004d50 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff43800bb00 con 0x7ff4380039f0 2024-04-25T11:21:06.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.170+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff43800bd00 con 0x7ff4500d12d0 2024-04-25T11:21:06.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44f7fe640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438004d50 0x7ff438005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44effd640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380039f0 0x7ff438003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44e7fc640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4500d12d0 0x7ff438004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44f7fe640 1 -- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4500d12d0 msgr2=0x7ff438004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:06.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44f7fe640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4500d12d0 0x7ff438004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44f7fe640 1 -- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380039f0 msgr2=0x7ff438003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:06.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44f7fe640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380039f0 0x7ff438003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44f7fe640 1 -- 172.21.15.106:0/241114186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff438003530 con 0x7ff438004d50 2024-04-25T11:21:06.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44f7fe640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438004d50 0x7ff438005160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff44800c4c0 tx=0x7ff44801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9c056be7e22ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:06.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44800c0e0 con 0x7ff438004d50 2024-04-25T11:21:06.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4480a9070 con 0x7ff438004d50 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44dffb640 1 -- 172.21.15.106:0/241114186 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480a5e00 con 0x7ff438004d50 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff45592c640 1 -- 172.21.15.106:0/241114186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438004d50 msgr2=0x7ff438005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff45592c640 1 --2- 172.21.15.106:0/241114186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438004d50 0x7ff438005160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff44800c4c0 tx=0x7ff44801cca0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff44f7fe640 1 -- 172.21.15.106:0/241114186 reap_dead start 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff45592c640 1 -- 172.21.15.106:0/241114186 shutdown_connections 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff45592c640 1 -- 172.21.15.106:0/241114186 >> 172.21.15.106:0/241114186 conn(0x7ff45006c7c0 msgr2=0x7ff45006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff45592c640 1 -- 172.21.15.106:0/241114186 shutdown_connections 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.174+0000 7ff45592c640 1 -- 172.21.15.106:0/241114186 wait complete. 2024-04-25T11:21:06.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 Processor -- start 2024-04-25T11:21:06.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 -- start start 2024-04-25T11:21:06.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438004d50 0x7ff450147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff450155990 0x7ff450155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff45015d090 0x7ff45015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff450160db0 con 0x7ff450155990 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff450160ad0 con 0x7ff438004d50 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff450161090 con 0x7ff45015d090 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff450155990 0x7ff450155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff450155990 0x7ff450155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36026/0 (socket says 172.21.15.106:36026) 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 -- 172.21.15.106:0/638218418 learned_addr learned my addr 172.21.15.106:0/638218418 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44effd640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438004d50 0x7ff450147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44f7fe640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff45015d090 0x7ff45015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 -- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff45015d090 msgr2=0x7ff45015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff45015d090 0x7ff45015d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 -- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438004d50 msgr2=0x7ff450147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438004d50 0x7ff450147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 -- 172.21.15.106:0/638218418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4501562e0 con 0x7ff450155990 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44effd640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438004d50 0x7ff450147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff44e7fc640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff450155990 0x7ff450155da0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff444011590 tx=0x7ff444011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff437fff640 1 -- 172.21.15.106:0/638218418 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444004220 con 0x7ff450155990 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff437fff640 1 -- 172.21.15.106:0/638218418 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4440043c0 con 0x7ff450155990 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff450156730 con 0x7ff450155990 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff437fff640 1 -- 172.21.15.106:0/638218418 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444015870 con 0x7ff450155990 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.178+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff450156cf0 con 0x7ff450155990 2024-04-25T11:21:06.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.182+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45015a7c0 con 0x7ff450155990 2024-04-25T11:21:06.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.182+0000 7ff437fff640 1 -- 172.21.15.106:0/638218418 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff444015a10 con 0x7ff450155990 2024-04-25T11:21:06.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.182+0000 7ff437fff640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff43c06f430 0x7ff43c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.182+0000 7ff44effd640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff43c06f430 0x7ff43c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.182+0000 7ff437fff640 1 -- 172.21.15.106:0/638218418 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ff44408f420 con 0x7ff450155990 2024-04-25T11:21:06.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.182+0000 7ff44effd640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff43c06f430 0x7ff43c071940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4500d1280 tx=0x7ff440009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:06.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.186+0000 7ff437fff640 1 -- 172.21.15.106:0/638218418 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff4501562e0 con 0x7ff450155990 2024-04-25T11:21:06.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.334+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4501564c0 con 0x7ff43c06f430 2024-04-25T11:21:06.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.334+0000 7ff437fff640 1 -- 172.21.15.106:0/638218418 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13511 (secure 0 0 0) 0x7ff4501564c0 con 0x7ff43c06f430 2024-04-25T11:21:06.336 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:21:06.336 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:21:06.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff43c06f430 msgr2=0x7ff43c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff43c06f430 0x7ff43c071940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4500d1280 tx=0x7ff440009040 comp rx=0 tx=0).stop 2024-04-25T11:21:06.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff450155990 msgr2=0x7ff450155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff450155990 0x7ff450155da0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff444011590 tx=0x7ff444011a50 comp rx=0 tx=0).stop 2024-04-25T11:21:06.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 shutdown_connections 2024-04-25T11:21:06.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff45015d090 0x7ff45015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff43c06f430 0x7ff43c071940 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff450155990 0x7ff450155da0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 --2- 172.21.15.106:0/638218418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438004d50 0x7ff450147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 >> 172.21.15.106:0/638218418 conn(0x7ff45006c7c0 msgr2=0x7ff4500b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:06.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 shutdown_connections 2024-04-25T11:21:06.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.338+0000 7ff45592c640 1 -- 172.21.15.106:0/638218418 wait complete. 2024-04-25T11:21:06.355 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-25T11:21:06.123738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":273748,"kb_used_data":57576,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754700972,"statfs":{"total":773094113280,"available":772813795328,"internally_reserved":0,"allocated":58957824,"data_stored":56987354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33570696,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-9,"num_read_kb":-8192,"num_write":-73,"num_write_kb":-32804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001704"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":138,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:21:02.688644+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:21:02.688644+0000","last_peered":"2024-04-25T11:21:02.688644+0000","last_clean":"2024-04-25T11:21:02.688644+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:21:02.688644+0000","last_undegraded":"2024-04-25T11:21:02.688644+0000","last_fullsized":"2024-04-25T11:21:02.688644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":5068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":5189632,"data_stored":4943364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43800,"kb_used_data":16776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328040,"statfs":{"total":96636764160,"available":96591912960,"internally_reserved":0,"allocated":17178624,"data_stored":16935908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43796,"kb_used_data":16776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328044,"statfs":{"total":96636764160,"available":96591917056,"internally_reserved":0,"allocated":17178624,"data_stored":16935908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":4488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":4595712,"data_stored":4352996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35604,"kb_used_data":8584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336236,"statfs":{"total":96636764160,"available":96600305664,"internally_reserved":0,"allocated":8790016,"data_stored":8547300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":5068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":5189632,"data_stored":4943364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:21:06.356 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:06.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fdaa571c640 1 Processor -- start 2024-04-25T11:21:06.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fdaa571c640 1 -- start start 2024-04-25T11:21:06.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fdaa571c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 0x7fdaa015db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fdaa571c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa00b5cc0 con 0x7fdaa015d6f0 2024-04-25T11:21:06.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fdaa571c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa00b5f80 con 0x7fdaa0160f10 2024-04-25T11:21:06.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fdaa571c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa00b6260 con 0x7fdaa00b4b00 2024-04-25T11:21:06.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda97fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdaa015d6f0 0x7fdaa00b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45874/0 (socket says 172.21.15.106:45874) 2024-04-25T11:21:06.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda97fff640 1 -- 172.21.15.106:0/2051464031 learned_addr learned my addr 172.21.15.106:0/2051464031 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:06.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9f7fe640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 0x7fdaa015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198694365 0 0) 0x7fdaa00b5cc0 con 0x7fdaa015d6f0 2024-04-25T11:21:06.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda8c003530 con 0x7fdaa015d6f0 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2213862895 0 0) 0x7fda8c003530 con 0x7fdaa015d6f0 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 msgr2=0x7fdaa015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 0x7fdaa015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 >> v1:172.21.15.134:6789/0 conn(0x7fdaa0160f10 legacy=0x7fdaa015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa00b6540 con 0x7fdaa015d6f0 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7fda90001680 con 0x7fdaa015d6f0 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 >> v1:172.21.15.106:6789/0 conn(0x7fdaa015d6f0 legacy=0x7fdaa00b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 0x7fda8c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c003f70 0x7fda8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda8c004d50 0x7fda8c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa00b6260 con 0x7fda8c004d50 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c00bc10 con 0x7fda8c003f70 2024-04-25T11:21:06.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.490+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c00be10 con 0x7fdaa00b4b00 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9f7fe640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda8c004d50 0x7fda8c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9effd640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 0x7fda8c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda97fff640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c003f70 0x7fda8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9f7fe640 1 -- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 msgr2=0x7fda8c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9f7fe640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 0x7fda8c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9f7fe640 1 -- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c003f70 msgr2=0x7fda8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9f7fe640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c003f70 0x7fda8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9f7fe640 1 -- 172.21.15.106:0/2051464031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c003530 con 0x7fda8c004d50 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9effd640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa00b4b00 0x7fda8c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:06.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda97fff640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c003f70 0x7fda8c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:06.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9f7fe640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda8c004d50 0x7fda8c005160 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fda98009f90 tx=0x7fda9801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321f10e3e2a0449e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:06.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda980b2070 con 0x7fda8c004d50 2024-04-25T11:21:06.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda980a5c70 con 0x7fda8c004d50 2024-04-25T11:21:06.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9e7fc640 1 -- 172.21.15.106:0/2051464031 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98009c10 con 0x7fda8c004d50 2024-04-25T11:21:06.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fdaa571c640 1 -- 172.21.15.106:0/2051464031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda8c004d50 msgr2=0x7fda8c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fdaa571c640 1 --2- 172.21.15.106:0/2051464031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda8c004d50 0x7fda8c005160 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fda98009f90 tx=0x7fda9801cc60 comp rx=0 tx=0).stop 2024-04-25T11:21:06.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fda9f7fe640 1 -- 172.21.15.106:0/2051464031 reap_dead start 2024-04-25T11:21:06.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fdaa571c640 1 -- 172.21.15.106:0/2051464031 shutdown_connections 2024-04-25T11:21:06.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fdaa571c640 1 -- 172.21.15.106:0/2051464031 >> 172.21.15.106:0/2051464031 conn(0x7fdaa006c7c0 msgr2=0x7fdaa006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:06.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fdaa571c640 1 -- 172.21.15.106:0/2051464031 shutdown_connections 2024-04-25T11:21:06.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.494+0000 7fdaa571c640 1 -- 172.21.15.106:0/2051464031 wait complete. 2024-04-25T11:21:06.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 Processor -- start 2024-04-25T11:21:06.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 -- start start 2024-04-25T11:21:06.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c004d50 0x7fdaa0140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa0155580 0x7fdaa0159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaa015a1c0 0x7fdaa0143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa015db00 con 0x7fdaa015a1c0 2024-04-25T11:21:06.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0161320 con 0x7fda8c004d50 2024-04-25T11:21:06.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa015dc80 con 0x7fdaa0155580 2024-04-25T11:21:06.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaa015a1c0 0x7fdaa0143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaa015a1c0 0x7fdaa0143410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36032/0 (socket says 172.21.15.106:36032) 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 -- 172.21.15.106:0/2915408858 learned_addr learned my addr 172.21.15.106:0/2915408858 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9effd640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c004d50 0x7fdaa0140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda97fff640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa0155580 0x7fdaa0159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 -- 172.21.15.106:0/2915408858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa0155580 msgr2=0x7fdaa0159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa0155580 0x7fdaa0159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 -- 172.21.15.106:0/2915408858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c004d50 msgr2=0x7fdaa0140850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c004d50 0x7fdaa0140850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 -- 172.21.15.106:0/2915408858 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa015a5d0 con 0x7fdaa015a1c0 2024-04-25T11:21:06.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda9f7fe640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaa015a1c0 0x7fdaa0143410 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fda98002240 tx=0x7fda98009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:06.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda977fe640 1 -- 172.21.15.106:0/2915408858 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda980b2040 con 0x7fdaa015a1c0 2024-04-25T11:21:06.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda977fe640 1 -- 172.21.15.106:0/2915408858 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda980a5a70 con 0x7fdaa015a1c0 2024-04-25T11:21:06.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa0157180 con 0x7fdaa015a1c0 2024-04-25T11:21:06.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fda977fe640 1 -- 172.21.15.106:0/2915408858 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98009c10 con 0x7fdaa015a1c0 2024-04-25T11:21:06.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.498+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa01542f0 con 0x7fdaa015a1c0 2024-04-25T11:21:06.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.502+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa0154060 con 0x7fdaa015a1c0 2024-04-25T11:21:06.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.502+0000 7fda977fe640 1 -- 172.21.15.106:0/2915408858 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda980be020 con 0x7fdaa015a1c0 2024-04-25T11:21:06.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.502+0000 7fda977fe640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda8006f400 0x7fda80071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:06.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.502+0000 7fda9effd640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda8006f400 0x7fda80071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:06.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.502+0000 7fda977fe640 1 -- 172.21.15.106:0/2915408858 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fda98129b30 con 0x7fdaa015a1c0 2024-04-25T11:21:06.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.502+0000 7fda9effd640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda8006f400 0x7fda80071910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda88000c10 tx=0x7fda88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:06.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.506+0000 7fda977fe640 1 -- 172.21.15.106:0/2915408858 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdaa015a5d0 con 0x7fdaa015a1c0 2024-04-25T11:21:06.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.658+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdaa006cf10 con 0x7fdaa015a1c0 2024-04-25T11:21:06.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.658+0000 7fda977fe640 1 -- 172.21.15.106:0/2915408858 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13058 (secure 0 0 0) 0x7fda980a5e20 con 0x7fdaa015a1c0 2024-04-25T11:21:06.659 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:21:06.659 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"fsid":"8efdb3a5-72cb-436f-9be5-cf4e40adf704","created":"2024-04-25T11:19:01.462716+0000","modified":"2024-04-25T11:21:03.499043+0000","last_up_change":"2024-04-25T11:19:13.270562+0000","last_in_change":"2024-04-25T11:19:04.152598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:19:14.141837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73ae35d4-be76-494e-9a4c-4787c1736653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6811","nonce":70365189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6814","nonce":70365189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6821","nonce":70365189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":70365189},{"type":"v1","addr":"172.21.15.106:6817","nonce":70365189}]},"public_addr":"172.21.15.106:6811/70365189","cluster_addr":"172.21.15.106:6814/70365189","heartbeat_back_addr":"172.21.15.106:6821/70365189","heartbeat_front_addr":"172.21.15.106:6817/70365189","state":["exists","up"]},{"osd":1,"uuid":"02b209ec-e13e-428f-a041-ef9f203b4c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6801","nonce":3323217021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6803","nonce":3323217021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6807","nonce":3323217021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3323217021},{"type":"v1","addr":"172.21.15.106:6805","nonce":3323217021}]},"public_addr":"172.21.15.106:6801/3323217021","cluster_addr":"172.21.15.106:6803/3323217021","heartbeat_back_addr":"172.21.15.106:6807/3323217021","heartbeat_front_addr":"172.21.15.106:6805/3323217021","state":["exists","up"]},{"osd":2,"uuid":"9f8f9520-a4aa-48db-b8fe-e3520a6b5f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6810","nonce":3117805672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6813","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6815","nonce":3117805672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6823","nonce":3117805672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3117805672},{"type":"v1","addr":"172.21.15.106:6820","nonce":3117805672}]},"public_addr":"172.21.15.106:6810/3117805672","cluster_addr":"172.21.15.106:6815/3117805672","heartbeat_back_addr":"172.21.15.106:6823/3117805672","heartbeat_front_addr":"172.21.15.106:6820/3117805672","state":["exists","up"]},{"osd":3,"uuid":"a1c3f77c-ad61-42ea-b742-611fb5c21cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6825","nonce":3914961038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6827","nonce":3914961038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6831","nonce":3914961038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3914961038},{"type":"v1","addr":"172.21.15.106:6829","nonce":3914961038}]},"public_addr":"172.21.15.106:6825/3914961038","cluster_addr":"172.21.15.106:6827/3914961038","heartbeat_back_addr":"172.21.15.106:6831/3914961038","heartbeat_front_addr":"172.21.15.106:6829/3914961038","state":["exists","up"]},{"osd":4,"uuid":"c1985b6a-49ee-4e63-af0e-5c5ebf138b90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6809","nonce":4246182191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6811","nonce":4246182191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6815","nonce":4246182191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4246182191},{"type":"v1","addr":"172.21.15.134:6813","nonce":4246182191}]},"public_addr":"172.21.15.134:6809/4246182191","cluster_addr":"172.21.15.134:6811/4246182191","heartbeat_back_addr":"172.21.15.134:6815/4246182191","heartbeat_front_addr":"172.21.15.134:6813/4246182191","state":["exists","up"]},{"osd":5,"uuid":"7ae5b812-55fc-41b9-8dfc-34108b145fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6801","nonce":1995022176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6803","nonce":1995022176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6807","nonce":1995022176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1995022176},{"type":"v1","addr":"172.21.15.134:6805","nonce":1995022176}]},"public_addr":"172.21.15.134:6801/1995022176","cluster_addr":"172.21.15.134:6803/1995022176","heartbeat_back_addr":"172.21.15.134:6807/1995022176","heartbeat_front_addr":"172.21.15.134:6805/1995022176","state":["exists","up"]},{"osd":6,"uuid":"b09bb1a2-485c-42f4-ac06-2f4895e19994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6825","nonce":2059200670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6827","nonce":2059200670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6831","nonce":2059200670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2059200670},{"type":"v1","addr":"172.21.15.134:6829","nonce":2059200670}]},"public_addr":"172.21.15.134:6825/2059200670","cluster_addr":"172.21.15.134:6827/2059200670","heartbeat_back_addr":"172.21.15.134:6831/2059200670","heartbeat_front_addr":"172.21.15.134:6829/2059200670","state":["exists","up"]},{"osd":7,"uuid":"245cbf8d-6718-4404-be12-ac042327c299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6817","nonce":1198352866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6819","nonce":1198352866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6823","nonce":1198352866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1198352866},{"type":"v1","addr":"172.21.15.134:6821","nonce":1198352866}]},"public_addr":"172.21.15.134:6817/1198352866","cluster_addr":"172.21.15.134:6819/1198352866","heartbeat_back_addr":"172.21.15.134:6823/1198352866","heartbeat_front_addr":"172.21.15.134:6821/1198352866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.622704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.787683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.803662+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.725933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.727033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.637295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.677109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:09.605506+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/1141169456":"2024-04-26T11:21:00.485171+0000","172.21.15.106:6836/1141169456":"2024-04-26T11:21:00.485171+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda8006f400 msgr2=0x7fda80071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda8006f400 0x7fda80071910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda88000c10 tx=0x7fda88009040 comp rx=0 tx=0).stop 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaa015a1c0 msgr2=0x7fdaa0143410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaa015a1c0 0x7fdaa0143410 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fda98002240 tx=0x7fda98009750 comp rx=0 tx=0).stop 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 shutdown_connections 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fda8006f400 0x7fda80071910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdaa015a1c0 0x7fdaa0143410 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaa0155580 0x7fdaa0159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 --2- 172.21.15.106:0/2915408858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c004d50 0x7fdaa0140850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 >> 172.21.15.106:0/2915408858 conn(0x7fdaa006c7c0 msgr2=0x7fdaa015f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:06.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 shutdown_connections 2024-04-25T11:21:06.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:06.662+0000 7fdaa571c640 1 -- 172.21.15.106:0/2915408858 wait complete. 2024-04-25T11:21:07.681 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T11:21:07.681 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:21:07.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e2463f640 1 Processor -- start 2024-04-25T11:21:07.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e2463f640 1 -- start start 2024-04-25T11:21:07.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e2463f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9e1c0b2f90 0x7f9e1c0b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e2463f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 0x7f9e1c06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e2463f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e1c06f530 con 0x7f9e1c160610 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e2463f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9e1c0b2df0 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e2463f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c15dc40 con 0x7f9e1c06ebe0 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e223db640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9e1c0b2f90 0x7f9e1c0b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e22bdc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9e1c160610 0x7f9e1c160a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45886/0 (socket says 172.21.15.106:45886) 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e22bdc640 1 -- 172.21.15.106:0/1984767744 learned_addr learned my addr 172.21.15.106:0/1984767744 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e21bda640 1 --2- 172.21.15.106:0/1984767744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 0x7f9e1c06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e223db640 1 -- 172.21.15.106:0/1984767744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 msgr2=0x7f9e1c06eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e223db640 1 --2- 172.21.15.106:0/1984767744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 0x7f9e1c06eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e213d9640 1 -- 172.21.15.106:0/1984767744 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2086858596 0 0) 0x7f9e140014a0 con 0x7f9e1c160610 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e223db640 1 -- 172.21.15.106:0/1984767744 >> v1:172.21.15.106:6789/0 conn(0x7f9e1c160610 legacy=0x7f9e1c160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.842+0000 7f9e223db640 1 -- 172.21.15.106:0/1984767744 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c0d0f60 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e223db640 1 --2- 172.21.15.106:0/1984767744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 0x7f9e1c0b28b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9e180097c0 tx=0x7f9e1801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb5989d0f9a384c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e213d9640 1 -- 172.21.15.106:0/1984767744 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180b0070 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e213d9640 1 -- 172.21.15.106:0/1984767744 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e180ab600 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e213d9640 1 -- 172.21.15.106:0/1984767744 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180ab9d0 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 -- 172.21.15.106:0/1984767744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 msgr2=0x7f9e1c0b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 --2- 172.21.15.106:0/1984767744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 0x7f9e1c0b28b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9e180097c0 tx=0x7f9e1801c410 comp rx=0 tx=0).stop 2024-04-25T11:21:07.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 -- 172.21.15.106:0/1984767744 shutdown_connections 2024-04-25T11:21:07.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 --2- 172.21.15.106:0/1984767744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 0x7f9e1c06eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 --2- 172.21.15.106:0/1984767744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 0x7f9e1c0b28b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 -- 172.21.15.106:0/1984767744 >> 172.21.15.106:0/1984767744 conn(0x7f9e1c068c40 msgr2=0x7f9e1c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:07.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 -- 172.21.15.106:0/1984767744 shutdown_connections 2024-04-25T11:21:07.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 -- 172.21.15.106:0/1984767744 wait complete. 2024-04-25T11:21:07.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 Processor -- start 2024-04-25T11:21:07.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.846+0000 7f9e2463f640 1 -- start start 2024-04-25T11:21:07.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 0x7f9e1c1638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 0x7f9e1c163e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e1c160610 0x7f9e1c164360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c16a340 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c16a1c0 con 0x7f9e1c160610 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c16a4c0 con 0x7f9e1c06ebe0 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 0x7f9e1c163e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 0x7f9e1c163e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36050/0 (socket says 172.21.15.106:36050) 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 -- 172.21.15.106:0/2381462127 learned_addr learned my addr 172.21.15.106:0/2381462127 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e22bdc640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e1c160610 0x7f9e1c164360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e223db640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 0x7f9e1c1638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 -- 172.21.15.106:0/2381462127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 msgr2=0x7f9e1c1638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e1c06ebe0 0x7f9e1c1638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 -- 172.21.15.106:0/2381462127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e1c160610 msgr2=0x7f9e1c164360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e1c160610 0x7f9e1c164360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 -- 172.21.15.106:0/2381462127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c15a2e0 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e22bdc640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e1c160610 0x7f9e1c164360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e21bda640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 0x7f9e1c163e20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00c820 tx=0x7f9e0c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c015070 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0c007a30 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c165db0 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c007e20 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c166370 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e1c1658f0 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e0c018070 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9e0006f400 0x7f9e00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f9e0c08f3c0 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e223db640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9e0006f400 0x7f9e00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7f9e00075310 0x7f9e00077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 --> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e00077e30 con 0x7f9e00075310 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9e0c08f850 con 0x7f9e1c0b2f90 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e223db640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9e0006f400 0x7f9e00071910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e180ad000 tx=0x7f9e18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.850+0000 7f9e22bdc640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7f9e00075310 0x7f9e00077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.854+0000 7f9e22bdc640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7f9e00075310 0x7f9e00077760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.854+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 <== osd.0 v2:172.21.15.106:6809/70365189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e00077e30 con 0x7f9e00075310 2024-04-25T11:21:07.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 --> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9e1c074520 con 0x7f9e00075310 2024-04-25T11:21:07.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e12ffd640 1 -- 172.21.15.106:0/2381462127 <== osd.0 v2:172.21.15.106:6809/70365189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9e1c074520 con 0x7f9e00075310 2024-04-25T11:21:07.867 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:21:07.867 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:21:07.867 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:21:07.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7f9e00075310 msgr2=0x7f9e00077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6809/70365189,v1:172.21.15.106:6811/70365189] conn(0x7f9e00075310 0x7f9e00077760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9e0006f400 msgr2=0x7f9e00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f9e0006f400 0x7f9e00071910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e180ad000 tx=0x7f9e18008040 comp rx=0 tx=0).stop 2024-04-25T11:21:07.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 msgr2=0x7f9e1c163e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 --2- 172.21.15.106:0/2381462127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e1c0b2f90 0x7f9e1c163e20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00c820 tx=0x7f9e0c00cce0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e22bdc640 1 -- 172.21.15.106:0/2381462127 reap_dead start 2024-04-25T11:21:07.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 shutdown_connections 2024-04-25T11:21:07.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 >> 172.21.15.106:0/2381462127 conn(0x7f9e1c068c40 msgr2=0x7f9e1c166d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:07.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 shutdown_connections 2024-04-25T11:21:07.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:07.866+0000 7f9e2463f640 1 -- 172.21.15.106:0/2381462127 wait complete. 2024-04-25T11:21:07.885 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T11:21:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.026+0000 7f289a245640 1 Processor -- start 2024-04-25T11:21:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.026+0000 7f289a245640 1 -- start start 2024-04-25T11:21:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f289a245640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2894160fc0 0x7f28940b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f289a245640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28940b5110 0x7f289406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f289a245640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28940b5520 con 0x7f289415cae0 2024-04-25T11:21:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f289a245640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f28940ce2a0 con 0x7f28940b5110 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f289a245640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f28940b4460 con 0x7f2894160fc0 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2893fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28940b5110 0x7f289406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2893fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28940b5110 0x7f289406c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36066/0 (socket says 172.21.15.106:36066) 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2893fff640 1 -- 172.21.15.106:0/2071401901 learned_addr learned my addr 172.21.15.106:0/2071401901 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f288bfff640 1 --2- 172.21.15.106:0/2071401901 >> v2:172.21.15.134:3300/0 conn(0x7f2894160fc0 0x7f28940b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2893fff640 1 -- 172.21.15.106:0/2071401901 >> v2:172.21.15.134:3300/0 conn(0x7f2894160fc0 msgr2=0x7f28940b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2893fff640 1 --2- 172.21.15.106:0/2071401901 >> v2:172.21.15.134:3300/0 conn(0x7f2894160fc0 0x7f28940b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2893fff640 1 -- 172.21.15.106:0/2071401901 >> v1:172.21.15.134:6789/0 conn(0x7f289415cae0 legacy=0x7f28940cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2893fff640 1 -- 172.21.15.106:0/2071401901 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28941613d0 con 0x7f28940b5110 2024-04-25T11:21:08.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2893fff640 1 --2- 172.21.15.106:0/2071401901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f289406c820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f289406d370 tx=0x7f288c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc87c2ba981794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2892ffd640 1 -- 172.21.15.106:0/2071401901 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0ac020 con 0x7f28940b5110 2024-04-25T11:21:08.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2892ffd640 1 -- 172.21.15.106:0/2071401901 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f288c0a6410 con 0x7f28940b5110 2024-04-25T11:21:08.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.030+0000 7f2892ffd640 1 -- 172.21.15.106:0/2071401901 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0a67e0 con 0x7f28940b5110 2024-04-25T11:21:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- 172.21.15.106:0/2071401901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 msgr2=0x7f289406c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 --2- 172.21.15.106:0/2071401901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f289406c820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f289406d370 tx=0x7f288c01c740 comp rx=0 tx=0).stop 2024-04-25T11:21:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- 172.21.15.106:0/2071401901 shutdown_connections 2024-04-25T11:21:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 --2- 172.21.15.106:0/2071401901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f289406c820 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 --2- 172.21.15.106:0/2071401901 >> v2:172.21.15.134:3300/0 conn(0x7f2894160fc0 0x7f28940b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- 172.21.15.106:0/2071401901 >> 172.21.15.106:0/2071401901 conn(0x7f28940b3560 msgr2=0x7f28940b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- 172.21.15.106:0/2071401901 shutdown_connections 2024-04-25T11:21:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- 172.21.15.106:0/2071401901 wait complete. 2024-04-25T11:21:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 Processor -- start 2024-04-25T11:21:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- start start 2024-04-25T11:21:08.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f2894143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289415cae0 0x7f28940af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2894160fc0 0x7f289415f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f289406d170 con 0x7f28940b5110 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28940b4740 con 0x7f289415cae0 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f289a245640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f28940cfde0 con 0x7f2894160fc0 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.034+0000 7f28937fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f2894143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f28937fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f2894143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36070/0 (socket says 172.21.15.106:36070) 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f28937fe640 1 -- 172.21.15.106:0/2720598474 learned_addr learned my addr 172.21.15.106:0/2720598474 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f288bfff640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289415cae0 0x7f28940af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f2893fff640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2894160fc0 0x7f289415f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f28937fe640 1 -- 172.21.15.106:0/2720598474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2894160fc0 msgr2=0x7f289415f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f28937fe640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2894160fc0 0x7f289415f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f28937fe640 1 -- 172.21.15.106:0/2720598474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289415cae0 msgr2=0x7f28940af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f28937fe640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289415cae0 0x7f28940af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f28937fe640 1 -- 172.21.15.106:0/2720598474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28940b04e0 con 0x7f28940b5110 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f28937fe640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f2894143d30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f288400b440 tx=0x7f288400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f2890ff9640 1 -- 172.21.15.106:0/2720598474 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884015020 con 0x7f28940b5110 2024-04-25T11:21:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f2890ff9640 1 -- 172.21.15.106:0/2720598474 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2884011890 con 0x7f28940b5110 2024-04-25T11:21:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28940aa800 con 0x7f28940b5110 2024-04-25T11:21:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f2890ff9640 1 -- 172.21.15.106:0/2720598474 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884011c80 con 0x7f28940b5110 2024-04-25T11:21:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28940aad80 con 0x7f28940b5110 2024-04-25T11:21:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2894050750 con 0x7f28940b5110 2024-04-25T11:21:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f2890ff9640 1 -- 172.21.15.106:0/2720598474 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2884018020 con 0x7f28940b5110 2024-04-25T11:21:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f2890ff9640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f287406f400 0x7f2874071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f288bfff640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f287406f400 0x7f2874071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f2890ff9640 1 -- 172.21.15.106:0/2720598474 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f288408f2a0 con 0x7f28940b5110 2024-04-25T11:21:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.038+0000 7f288bfff640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f287406f400 0x7f2874071910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2894167940 tx=0x7f287c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.042+0000 7f2890ff9640 1 -- 172.21.15.106:0/2720598474 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f288401f030 con 0x7f28940b5110 2024-04-25T11:21:08.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.198+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f28940b6780 con 0x7f287406f400 2024-04-25T11:21:08.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.198+0000 7f2890ff9640 1 -- 172.21.15.106:0/2720598474 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f28940b6780 con 0x7f287406f400 2024-04-25T11:21:08.200 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T11:21:08.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f287406f400 msgr2=0x7f2874071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f287406f400 0x7f2874071910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2894167940 tx=0x7f287c008040 comp rx=0 tx=0).stop 2024-04-25T11:21:08.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 msgr2=0x7f2894143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f2894143d30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f288400b440 tx=0x7f288400b900 comp rx=0 tx=0).stop 2024-04-25T11:21:08.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 shutdown_connections 2024-04-25T11:21:08.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2894160fc0 0x7f289415f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289415cae0 0x7f28940af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f287406f400 0x7f2874071910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 --2- 172.21.15.106:0/2720598474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28940b5110 0x7f2894143d30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 >> 172.21.15.106:0/2720598474 conn(0x7f28940b3560 msgr2=0x7f28940cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 shutdown_connections 2024-04-25T11:21:08.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.202+0000 7f289a245640 1 -- 172.21.15.106:0/2720598474 wait complete. 2024-04-25T11:21:08.220 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T11:21:08.220 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:21:08.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.370+0000 7fd791422640 1 Processor -- start 2024-04-25T11:21:08.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.370+0000 7fd791422640 1 -- start start 2024-04-25T11:21:08.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd791422640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd78c154d40 0x7fd78c1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd791422640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd78c0c69f0 0x7fd78c0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd791422640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78c155150 con 0x7fd78c0d0400 2024-04-25T11:21:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd791422640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd78c15e610 con 0x7fd78c0c69f0 2024-04-25T11:21:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd791422640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd78c15e8f0 con 0x7fd78c154d40 2024-04-25T11:21:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78a7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd78c0d0400 0x7fd78c0d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45898/0 (socket says 172.21.15.106:45898) 2024-04-25T11:21:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78a7fc640 1 -- 172.21.15.106:0/2796622736 learned_addr learned my addr 172.21.15.106:0/2796622736 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78b7fe640 1 --2- 172.21.15.106:0/2796622736 >> v2:172.21.15.106:3300/0 conn(0x7fd78c0c69f0 0x7fd78c0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78affd640 1 --2- 172.21.15.106:0/2796622736 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd78c154d40 0x7fd78c1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd789ffb640 1 -- 172.21.15.106:0/2796622736 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4290630263 0 0) 0x7fd78c155150 con 0x7fd78c0d0400 2024-04-25T11:21:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd789ffb640 1 -- 172.21.15.106:0/2796622736 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd774003530 con 0x7fd78c0d0400 2024-04-25T11:21:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78b7fe640 1 -- 172.21.15.106:0/2796622736 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd78c154d40 msgr2=0x7fd78c1622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78b7fe640 1 --2- 172.21.15.106:0/2796622736 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd78c154d40 0x7fd78c1622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78b7fe640 1 -- 172.21.15.106:0/2796622736 >> v1:172.21.15.106:6789/0 conn(0x7fd78c0d0400 legacy=0x7fd78c0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78b7fe640 1 -- 172.21.15.106:0/2796622736 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c15ebd0 con 0x7fd78c0c69f0 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.374+0000 7fd78b7fe640 1 --2- 172.21.15.106:0/2796622736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0c69f0 0x7fd78c0c6e00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd784009d90 tx=0x7fd78401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6375aa0476052111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd789ffb640 1 -- 172.21.15.106:0/2796622736 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7840b2070 con 0x7fd78c0c69f0 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd789ffb640 1 -- 172.21.15.106:0/2796622736 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7840a5870 con 0x7fd78c0c69f0 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd789ffb640 1 -- 172.21.15.106:0/2796622736 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7840a5c40 con 0x7fd78c0c69f0 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 -- 172.21.15.106:0/2796622736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0c69f0 msgr2=0x7fd78c0c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 --2- 172.21.15.106:0/2796622736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0c69f0 0x7fd78c0c6e00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd784009d90 tx=0x7fd78401c8e0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 -- 172.21.15.106:0/2796622736 shutdown_connections 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 --2- 172.21.15.106:0/2796622736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0c69f0 0x7fd78c0c6e00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 --2- 172.21.15.106:0/2796622736 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd78c154d40 0x7fd78c1622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 -- 172.21.15.106:0/2796622736 >> 172.21.15.106:0/2796622736 conn(0x7fd78c068c40 msgr2=0x7fd78c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 -- 172.21.15.106:0/2796622736 shutdown_connections 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 -- 172.21.15.106:0/2796622736 wait complete. 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 Processor -- start 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.378+0000 7fd791422640 1 -- start start 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd78c0c69f0 0x7fd78c142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0d0400 0x7fd78c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd78c154d40 0x7fd78c0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c06ed60 con 0x7fd78c0d0400 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c06ebe0 con 0x7fd78c0c69f0 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd78c16aa40 con 0x7fd78c154d40 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0d0400 0x7fd78c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0d0400 0x7fd78c153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36076/0 (socket says 172.21.15.106:36076) 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 -- 172.21.15.106:0/4256365646 learned_addr learned my addr 172.21.15.106:0/4256365646 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 -- 172.21.15.106:0/4256365646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd78c154d40 msgr2=0x7fd78c0b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd78c154d40 0x7fd78c0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 -- 172.21.15.106:0/4256365646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd78c0c69f0 msgr2=0x7fd78c142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78affd640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd78c0c69f0 0x7fd78c142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd78c0c69f0 0x7fd78c142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 -- 172.21.15.106:0/4256365646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c154750 con 0x7fd78c0d0400 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78a7fc640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0d0400 0x7fd78c153a20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd78000c3f0 tx=0x7fd78000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780017070 con 0x7fd78c0d0400 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd780005cf0 con 0x7fd78c0d0400 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c14c4d0 con 0x7fd78c0d0400 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800072f0 con 0x7fd78c0d0400 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c14b9f0 con 0x7fd78c0d0400 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd78c074520 con 0x7fd78c0d0400 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd78001a070 con 0x7fd78c0d0400 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd77806f400 0x7fd778071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fd78008eb10 con 0x7fd78c0d0400 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78affd640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd77806f400 0x7fd778071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7fd778075310 0x7fd778077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 --> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd778077e30 con 0x7fd778075310 2024-04-25T11:21:08.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd78008efa0 con 0x7fd78c0d0400 2024-04-25T11:21:08.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78b7fe640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7fd778075310 0x7fd778077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.382+0000 7fd78affd640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd77806f400 0x7fd778071910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0b6e70 tx=0x7fd77c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.386+0000 7fd78b7fe640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7fd778075310 0x7fd778077760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.386+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 <== osd.1 v2:172.21.15.106:6800/3323217021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd778077e30 con 0x7fd778075310 2024-04-25T11:21:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 --> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd78c154500 con 0x7fd778075310 2024-04-25T11:21:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd7737fe640 1 -- 172.21.15.106:0/4256365646 <== osd.1 v2:172.21.15.106:6800/3323217021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd78c154500 con 0x7fd778075310 2024-04-25T11:21:08.399 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:21:08.399 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:21:08.399 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7fd778075310 msgr2=0x7fd778077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6800/3323217021,v1:172.21.15.106:6801/3323217021] conn(0x7fd778075310 0x7fd778077760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd77806f400 msgr2=0x7fd778071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fd77806f400 0x7fd778071910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0b6e70 tx=0x7fd77c008040 comp rx=0 tx=0).stop 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0d0400 msgr2=0x7fd78c153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 --2- 172.21.15.106:0/4256365646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c0d0400 0x7fd78c153a20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd78000c3f0 tx=0x7fd78000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd78b7fe640 1 -- 172.21.15.106:0/4256365646 reap_dead start 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 shutdown_connections 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 >> 172.21.15.106:0/4256365646 conn(0x7fd78c068c40 msgr2=0x7fd78c160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 shutdown_connections 2024-04-25T11:21:08.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.398+0000 7fd791422640 1 -- 172.21.15.106:0/4256365646 wait complete. 2024-04-25T11:21:08.417 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T11:21:08.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6ce24640 1 Processor -- start 2024-04-25T11:21:08.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6ce24640 1 -- start start 2024-04-25T11:21:08.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6ce24640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8b6806c820 0x7f8b6806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6ce24640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b6806f7e0 0x7f8b6806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6ce24640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b6806d170 con 0x7f8b68073460 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6ce24640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8b6815f1a0 con 0x7f8b6806f7e0 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6ce24640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8b68072fd0 con 0x7f8b6806c820 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b66d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b6806f7e0 0x7f8b6806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8b68073460 0x7f8b6815e190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45906/0 (socket says 172.21.15.106:45906) 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b6659c640 1 -- 172.21.15.106:0/3435673597 learned_addr learned my addr 172.21.15.106:0/3435673597 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.558+0000 7f8b5dd9b640 1 --2- 172.21.15.106:0/3435673597 >> v2:172.21.15.134:3300/0 conn(0x7f8b6806c820 0x7f8b6806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b66d9d640 1 -- 172.21.15.106:0/3435673597 >> v2:172.21.15.134:3300/0 conn(0x7f8b6806c820 msgr2=0x7f8b6806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b66d9d640 1 --2- 172.21.15.106:0/3435673597 >> v2:172.21.15.134:3300/0 conn(0x7f8b6806c820 0x7f8b6806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b66d9d640 1 -- 172.21.15.106:0/3435673597 >> v1:172.21.15.106:6789/0 conn(0x7f8b68073460 legacy=0x7f8b6815e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b66d9d640 1 -- 172.21.15.106:0/3435673597 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b68144350 con 0x7f8b6806f7e0 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b65d9b640 1 -- 172.21.15.106:0/3435673597 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1504851233 0 0) 0x7f8b58001120 con 0x7f8b68073460 2024-04-25T11:21:08.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b5dd9b640 1 --2- 172.21.15.106:0/3435673597 >> v2:172.21.15.134:3300/0 conn(0x7f8b6806c820 0x7f8b6806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:08.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b66d9d640 1 --2- 172.21.15.106:0/3435673597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6806f7e0 0x7f8b6806fbf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8b680b4510 tx=0x7f8b6001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed7fcc610cb23c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b65d9b640 1 -- 172.21.15.106:0/3435673597 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600b2070 con 0x7f8b6806f7e0 2024-04-25T11:21:08.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b65d9b640 1 -- 172.21.15.106:0/3435673597 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b600a59c0 con 0x7f8b6806f7e0 2024-04-25T11:21:08.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b65d9b640 1 -- 172.21.15.106:0/3435673597 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600a5d90 con 0x7f8b6806f7e0 2024-04-25T11:21:08.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b6ce24640 1 -- 172.21.15.106:0/3435673597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6806f7e0 msgr2=0x7f8b6806fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/3435673597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6806f7e0 0x7f8b6806fbf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8b680b4510 tx=0x7f8b6001c900 comp rx=0 tx=0).stop 2024-04-25T11:21:08.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b6ce24640 1 -- 172.21.15.106:0/3435673597 shutdown_connections 2024-04-25T11:21:08.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/3435673597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6806f7e0 0x7f8b6806fbf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/3435673597 >> v2:172.21.15.134:3300/0 conn(0x7f8b6806c820 0x7f8b6806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b6ce24640 1 -- 172.21.15.106:0/3435673597 >> 172.21.15.106:0/3435673597 conn(0x7f8b680b3560 msgr2=0x7f8b680b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b6ce24640 1 -- 172.21.15.106:0/3435673597 shutdown_connections 2024-04-25T11:21:08.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.562+0000 7f8b6ce24640 1 -- 172.21.15.106:0/3435673597 wait complete. 2024-04-25T11:21:08.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 Processor -- start 2024-04-25T11:21:08.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 -- start start 2024-04-25T11:21:08.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b6806c820 0x7f8b6815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6806f7e0 0x7f8b6815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b68073460 0x7f8b68133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68073870 con 0x7f8b68073460 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6815e8a0 con 0x7f8b6806f7e0 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8b680739f0 con 0x7f8b6806c820 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b68073460 0x7f8b68133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b68073460 0x7f8b68133830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36098/0 (socket says 172.21.15.106:36098) 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 -- 172.21.15.106:0/2471517200 learned_addr learned my addr 172.21.15.106:0/2471517200 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b5dd9b640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6806f7e0 0x7f8b6815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6659c640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b6806c820 0x7f8b6815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 -- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b6806c820 msgr2=0x7f8b6815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b6806c820 0x7f8b6815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 -- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6806f7e0 msgr2=0x7f8b6815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6806f7e0 0x7f8b6815c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 -- 172.21.15.106:0/2471517200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b681367f0 con 0x7f8b68073460 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6659c640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b6806c820 0x7f8b6815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b66d9d640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b68073460 0x7f8b68133830 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8b600b0040 tx=0x7f8b600196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2471517200 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600bd020 con 0x7f8b68073460 2024-04-25T11:21:08.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2471517200 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b600ad740 con 0x7f8b68073460 2024-04-25T11:21:08.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6815cf60 con 0x7f8b68073460 2024-04-25T11:21:08.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2471517200 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600b2040 con 0x7f8b68073460 2024-04-25T11:21:08.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.566+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6815d520 con 0x7f8b68073460 2024-04-25T11:21:08.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.570+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6806f390 con 0x7f8b68073460 2024-04-25T11:21:08.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.570+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2471517200 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8b600c0020 con 0x7f8b68073460 2024-04-25T11:21:08.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.570+0000 7f8b5f7fe640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8b4806f400 0x7f8b48071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.570+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2471517200 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f8b6012a0b0 con 0x7f8b68073460 2024-04-25T11:21:08.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.570+0000 7f8b6659c640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8b4806f400 0x7f8b48071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.570+0000 7f8b6659c640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8b4806f400 0x7f8b48071910 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8b58007fd0 tx=0x7f8b58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.574+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2471517200 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8b600c7030 con 0x7f8b68073460 2024-04-25T11:21:08.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.734+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f8b68161090 con 0x7f8b4806f400 2024-04-25T11:21:08.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.734+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2471517200 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8b68161090 con 0x7f8b4806f400 2024-04-25T11:21:08.735 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T11:21:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8b4806f400 msgr2=0x7f8b48071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8b4806f400 0x7f8b48071910 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8b58007fd0 tx=0x7f8b58009040 comp rx=0 tx=0).stop 2024-04-25T11:21:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b68073460 msgr2=0x7f8b68133830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b68073460 0x7f8b68133830 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8b600b0040 tx=0x7f8b600196a0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 shutdown_connections 2024-04-25T11:21:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f8b4806f400 0x7f8b48071910 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b68073460 0x7f8b68133830 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6806f7e0 0x7f8b6815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 --2- 172.21.15.106:0/2471517200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b6806c820 0x7f8b6815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 >> 172.21.15.106:0/2471517200 conn(0x7f8b680b3560 msgr2=0x7f8b6815f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 shutdown_connections 2024-04-25T11:21:08.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.738+0000 7f8b6ce24640 1 -- 172.21.15.106:0/2471517200 wait complete. 2024-04-25T11:21:08.756 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T11:21:08.757 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:21:08.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76fb5b640 1 Processor -- start 2024-04-25T11:21:08.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76fb5b640 1 -- start start 2024-04-25T11:21:08.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76fb5b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76fb5b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe76815c2e0 0x7fe76815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76fb5b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe768073530 con 0x7fe768073b70 2024-04-25T11:21:08.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76fb5b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe7680cf270 con 0x7fe76815c2e0 2024-04-25T11:21:08.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76fb5b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe7680cf550 con 0x7fe768072be0 2024-04-25T11:21:08.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76d0f6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76e0f8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe76815c2e0 0x7fe76815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76d0f6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:52852/0 (socket says 172.21.15.106:52852) 2024-04-25T11:21:08.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76d8f7640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe768073b70 0x7fe7680d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:39094/0 (socket says 172.21.15.106:39094) 2024-04-25T11:21:08.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76d0f6640 1 -- 172.21.15.106:0/2528346397 learned_addr learned my addr 172.21.15.106:0/2528346397 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:08.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76d0f6640 1 -- 172.21.15.106:0/2528346397 >> v2:172.21.15.134:3300/0 conn(0x7fe76815c2e0 msgr2=0x7fe76815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:08.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76d0f6640 1 --2- 172.21.15.106:0/2528346397 >> v2:172.21.15.134:3300/0 conn(0x7fe76815c2e0 0x7fe76815c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76d0f6640 1 -- 172.21.15.106:0/2528346397 >> v1:172.21.15.134:6789/0 conn(0x7fe768073b70 legacy=0x7fe7680d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76d0f6640 1 -- 172.21.15.106:0/2528346397 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7680cf830 con 0x7fe768072be0 2024-04-25T11:21:08.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.898+0000 7fe76e0f8640 1 --2- 172.21.15.106:0/2528346397 >> v2:172.21.15.134:3300/0 conn(0x7fe76815c2e0 0x7fe76815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:21:08.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76d0f6640 1 --2- 172.21.15.106:0/2528346397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768072ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe758009870 tx=0x7fe75801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934c7a4236adc15c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76c8f5640 1 -- 172.21.15.106:0/2528346397 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580ab5f0 con 0x7fe768072be0 2024-04-25T11:21:08.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76c8f5640 1 -- 172.21.15.106:0/2528346397 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7580b0070 con 0x7fe768072be0 2024-04-25T11:21:08.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76c8f5640 1 -- 172.21.15.106:0/2528346397 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580ab9b0 con 0x7fe768072be0 2024-04-25T11:21:08.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76fb5b640 1 -- 172.21.15.106:0/2528346397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 msgr2=0x7fe768072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76fb5b640 1 --2- 172.21.15.106:0/2528346397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768072ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe758009870 tx=0x7fe75801c560 comp rx=0 tx=0).stop 2024-04-25T11:21:08.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76fb5b640 1 -- 172.21.15.106:0/2528346397 shutdown_connections 2024-04-25T11:21:08.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76fb5b640 1 --2- 172.21.15.106:0/2528346397 >> v2:172.21.15.134:3300/0 conn(0x7fe76815c2e0 0x7fe76815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76fb5b640 1 --2- 172.21.15.106:0/2528346397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768072ff0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76fb5b640 1 -- 172.21.15.106:0/2528346397 >> 172.21.15.106:0/2528346397 conn(0x7fe768068c40 msgr2=0x7fe76806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76fb5b640 1 -- 172.21.15.106:0/2528346397 shutdown_connections 2024-04-25T11:21:08.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.902+0000 7fe76fb5b640 1 -- 172.21.15.106:0/2528346397 wait complete. 2024-04-25T11:21:08.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76fb5b640 1 Processor -- start 2024-04-25T11:21:08.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76fb5b640 1 -- start start 2024-04-25T11:21:08.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76fb5b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76fb5b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe768073b70 0x7fe768136d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76fb5b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe76815c2e0 0x7fe76815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76fb5b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe76815d7d0 con 0x7fe768073b70 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76fb5b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe76815d650 con 0x7fe76815c2e0 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76fb5b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe768166ed0 con 0x7fe768072be0 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76d0f6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe768073b70 0x7fe768136d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76d0f6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe768073b70 0x7fe768136d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36104/0 (socket says 172.21.15.106:36104) 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76d0f6640 1 -- 172.21.15.106:0/4087552809 learned_addr learned my addr 172.21.15.106:0/4087552809 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76d8f7640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.906+0000 7fe76e0f8640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe76815c2e0 0x7fe76815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76d0f6640 1 -- 172.21.15.106:0/4087552809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 msgr2=0x7fe768144df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76d0f6640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe768072be0 0x7fe768144df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76d0f6640 1 -- 172.21.15.106:0/4087552809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe76815c2e0 msgr2=0x7fe76815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76d0f6640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe76815c2e0 0x7fe76815ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76d0f6640 1 -- 172.21.15.106:0/4087552809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe768145640 con 0x7fe768073b70 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76d0f6640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe768073b70 0x7fe768136d60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe75801ac10 tx=0x7fe75801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580be5f0 con 0x7fe768073b70 2024-04-25T11:21:08.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7580be790 con 0x7fe768073b70 2024-04-25T11:21:08.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe768145920 con 0x7fe768073b70 2024-04-25T11:21:08.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580beb60 con 0x7fe768073b70 2024-04-25T11:21:08.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe768145ee0 con 0x7fe768073b70 2024-04-25T11:21:08.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe768146120 con 0x7fe768073b70 2024-04-25T11:21:08.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7580c8430 con 0x7fe768073b70 2024-04-25T11:21:08.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe74c06f400 0x7fe74c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76d8f7640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe74c06f400 0x7fe74c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fe758129900 con 0x7fe768073b70 2024-04-25T11:21:08.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7fe74c075310 0x7fe74c077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 --> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe75800af30 con 0x7fe74c075310 2024-04-25T11:21:08.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe758129d90 con 0x7fe768073b70 2024-04-25T11:21:08.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76e0f8640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7fe74c075310 0x7fe74c077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76d8f7640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe74c06f400 0x7fe74c071910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe764000f70 tx=0x7fe764007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.910+0000 7fe76e0f8640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7fe74c075310 0x7fe74c077760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.914+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 <== osd.2 v2:172.21.15.106:6808/3117805672 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe75800af30 con 0x7fe74c075310 2024-04-25T11:21:08.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 --> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe7681453f0 con 0x7fe74c075310 2024-04-25T11:21:08.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe75e7fc640 1 -- 172.21.15.106:0/4087552809 <== osd.2 v2:172.21.15.106:6808/3117805672 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe7681453f0 con 0x7fe74c075310 2024-04-25T11:21:08.926 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:21:08.926 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:21:08.926 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7fe74c075310 msgr2=0x7fe74c077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6808/3117805672,v1:172.21.15.106:6810/3117805672] conn(0x7fe74c075310 0x7fe74c077760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe74c06f400 msgr2=0x7fe74c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fe74c06f400 0x7fe74c071910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe764000f70 tx=0x7fe764007cf0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe768073b70 msgr2=0x7fe768136d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 --2- 172.21.15.106:0/4087552809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe768073b70 0x7fe768136d60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe75801ac10 tx=0x7fe75801ada0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76e0f8640 1 -- 172.21.15.106:0/4087552809 reap_dead start 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 shutdown_connections 2024-04-25T11:21:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 >> 172.21.15.106:0/4087552809 conn(0x7fe768068c40 msgr2=0x7fe7680b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 shutdown_connections 2024-04-25T11:21:08.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:08.926+0000 7fe76fb5b640 1 -- 172.21.15.106:0/4087552809 wait complete. 2024-04-25T11:21:08.943 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T11:21:09.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.082+0000 7ffb69a4e640 1 Processor -- start 2024-04-25T11:21:09.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.082+0000 7ffb69a4e640 1 -- start start 2024-04-25T11:21:09.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb69a4e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb64073460 0x7ffb6415e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb69a4e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffb6415e6d0 0x7ffb6406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb69a4e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffb6406cd60 0x7ffb6406f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb69a4e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ffb6415f1a0 con 0x7ffb6406cd60 2024-04-25T11:21:09.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb69a4e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ffb64072fd0 con 0x7ffb6415e6d0 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb69a4e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6406d170 con 0x7ffb64073460 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffb6406cd60 0x7ffb6406f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffb6406cd60 0x7ffb6406f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36118/0 (socket says 172.21.15.106:36118) 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 -- 172.21.15.106:0/3980736137 learned_addr learned my addr 172.21.15.106:0/3980736137 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb62ffd640 1 --2- 172.21.15.106:0/3980736137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb64073460 0x7ffb6415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb5a7fc640 1 --2- 172.21.15.106:0/3980736137 >> v2:172.21.15.134:3300/0 conn(0x7ffb6415e6d0 0x7ffb6406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 -- 172.21.15.106:0/3980736137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb64073460 msgr2=0x7ffb6415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 --2- 172.21.15.106:0/3980736137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb64073460 0x7ffb6415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 -- 172.21.15.106:0/3980736137 >> v2:172.21.15.134:3300/0 conn(0x7ffb6415e6d0 msgr2=0x7ffb6406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 --2- 172.21.15.106:0/3980736137 >> v2:172.21.15.134:3300/0 conn(0x7ffb6415e6d0 0x7ffb6406c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 -- 172.21.15.106:0/3980736137 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb64144350 con 0x7ffb6406cd60 2024-04-25T11:21:09.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb637fe640 1 --2- 172.21.15.106:0/3980736137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6406cd60 0x7ffb6406f7e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffb640b6830 tx=0x7ffb5c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd37f28c934db72a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb627fc640 1 -- 172.21.15.106:0/3980736137 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0a5840 con 0x7ffb6406cd60 2024-04-25T11:21:09.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb627fc640 1 -- 172.21.15.106:0/3980736137 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0a59e0 con 0x7ffb6406cd60 2024-04-25T11:21:09.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.086+0000 7ffb627fc640 1 -- 172.21.15.106:0/3980736137 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0a5db0 con 0x7ffb6406cd60 2024-04-25T11:21:09.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3980736137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6406cd60 msgr2=0x7ffb6406f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3980736137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6406cd60 0x7ffb6406f7e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffb640b6830 tx=0x7ffb5c01c9c0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3980736137 shutdown_connections 2024-04-25T11:21:09.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3980736137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6406cd60 0x7ffb6406f7e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3980736137 >> v2:172.21.15.134:3300/0 conn(0x7ffb6415e6d0 0x7ffb6406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3980736137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb64073460 0x7ffb6415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3980736137 >> 172.21.15.106:0/3980736137 conn(0x7ffb640b3560 msgr2=0x7ffb640b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3980736137 shutdown_connections 2024-04-25T11:21:09.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3980736137 wait complete. 2024-04-25T11:21:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 Processor -- start 2024-04-25T11:21:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- start start 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb6406cd60 0x7ffb6413e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64073460 0x7ffb6415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6415e6d0 0x7ffb64161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffb640739f0 con 0x7ffb6415e6d0 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64073870 con 0x7ffb64073460 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb69a4e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffb64070130 con 0x7ffb6406cd60 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6415e6d0 0x7ffb64161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6415e6d0 0x7ffb64161f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36134/0 (socket says 172.21.15.106:36134) 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 -- 172.21.15.106:0/3898206188 learned_addr learned my addr 172.21.15.106:0/3898206188 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb5a7fc640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64073460 0x7ffb6415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb62ffd640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb6406cd60 0x7ffb6413e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 -- 172.21.15.106:0/3898206188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb6406cd60 msgr2=0x7ffb6413e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb6406cd60 0x7ffb6413e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 -- 172.21.15.106:0/3898206188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64073460 msgr2=0x7ffb6415ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64073460 0x7ffb6415ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 -- 172.21.15.106:0/3898206188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb64139920 con 0x7ffb6415e6d0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.090+0000 7ffb637fe640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6415e6d0 0x7ffb64161f90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c01b070 tx=0x7ffb5c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb60ff9640 1 -- 172.21.15.106:0/3898206188 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0b8670 con 0x7ffb6415e6d0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb60ff9640 1 -- 172.21.15.106:0/3898206188 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0be070 con 0x7ffb6415e6d0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb64139c00 con 0x7ffb6415e6d0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb60ff9640 1 -- 172.21.15.106:0/3898206188 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0b8a90 con 0x7ffb6415e6d0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6413a1c0 con 0x7ffb6415e6d0 2024-04-25T11:21:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6406f390 con 0x7ffb6415e6d0 2024-04-25T11:21:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb60ff9640 1 -- 172.21.15.106:0/3898206188 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffb5c0b8c30 con 0x7ffb6415e6d0 2024-04-25T11:21:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb60ff9640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffb4406f400 0x7ffb44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb62ffd640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffb4406f400 0x7ffb44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb60ff9640 1 -- 172.21.15.106:0/3898206188 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ffb5c12a1b0 con 0x7ffb6415e6d0 2024-04-25T11:21:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.094+0000 7ffb62ffd640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffb4406f400 0x7ffb44071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffb64070ae0 tx=0x7ffb54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.098+0000 7ffb60ff9640 1 -- 172.21.15.106:0/3898206188 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb5c0165e0 con 0x7ffb6415e6d0 2024-04-25T11:21:09.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.246+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7ffb641471f0 con 0x7ffb4406f400 2024-04-25T11:21:09.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb60ff9640 1 -- 172.21.15.106:0/3898206188 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffb641471f0 con 0x7ffb4406f400 2024-04-25T11:21:09.248 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffb4406f400 msgr2=0x7ffb44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffb4406f400 0x7ffb44071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffb64070ae0 tx=0x7ffb54008040 comp rx=0 tx=0).stop 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6415e6d0 msgr2=0x7ffb64161f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6415e6d0 0x7ffb64161f90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c01b070 tx=0x7ffb5c019760 comp rx=0 tx=0).stop 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 shutdown_connections 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ffb4406f400 0x7ffb44071910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb6415e6d0 0x7ffb64161f90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64073460 0x7ffb6415ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 --2- 172.21.15.106:0/3898206188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb6406cd60 0x7ffb6413e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 >> 172.21.15.106:0/3898206188 conn(0x7ffb640b3560 msgr2=0x7ffb64146140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 shutdown_connections 2024-04-25T11:21:09.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.250+0000 7ffb69a4e640 1 -- 172.21.15.106:0/3898206188 wait complete. 2024-04-25T11:21:09.269 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T11:21:09.269 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:21:09.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f758a30b640 1 Processor -- start 2024-04-25T11:21:09.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f758a30b640 1 -- start start 2024-04-25T11:21:09.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f758a30b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f75840d0b10 0x7f75841622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f758a30b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75840c69f0 0x7f75840c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f758a30b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7584155150 con 0x7f7584154d40 2024-04-25T11:21:09.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f758a30b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f758415e610 con 0x7f75840c69f0 2024-04-25T11:21:09.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f758a30b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f758415e8f0 con 0x7f75840d0b10 2024-04-25T11:21:09.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f7583fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7584154d40 0x7f75840d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45908/0 (socket says 172.21.15.106:45908) 2024-04-25T11:21:09.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f75888a8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75840c69f0 0x7f75840c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.414+0000 7f7583fff640 1 -- 172.21.15.106:0/1922408904 learned_addr learned my addr 172.21.15.106:0/1922408904 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:09.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75888a8640 1 -- 172.21.15.106:0/1922408904 >> v2:172.21.15.134:3300/0 conn(0x7f75840d0b10 msgr2=0x7f75841622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:09.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75888a8640 1 --2- 172.21.15.106:0/1922408904 >> v2:172.21.15.134:3300/0 conn(0x7f75840d0b10 0x7f75841622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75888a8640 1 -- 172.21.15.106:0/1922408904 >> v1:172.21.15.106:6789/0 conn(0x7f7584154d40 legacy=0x7f75840d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75888a8640 1 -- 172.21.15.106:0/1922408904 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758415ebd0 con 0x7f75840c69f0 2024-04-25T11:21:09.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75837fe640 1 -- 172.21.15.106:0/1922408904 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 576406637 0 0) 0x7f7584155150 con 0x7f7584154d40 2024-04-25T11:21:09.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75888a8640 1 --2- 172.21.15.106:0/1922408904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75840c69f0 0x7f75840c6e00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7584070e40 tx=0x7f757c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54cfa4f0c5a331e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75837fe640 1 -- 172.21.15.106:0/1922408904 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0b2070 con 0x7f75840c69f0 2024-04-25T11:21:09.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75837fe640 1 -- 172.21.15.106:0/1922408904 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c0a5890 con 0x7f75840c69f0 2024-04-25T11:21:09.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f75837fe640 1 -- 172.21.15.106:0/1922408904 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0a5c60 con 0x7f75840c69f0 2024-04-25T11:21:09.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f758a30b640 1 -- 172.21.15.106:0/1922408904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75840c69f0 msgr2=0x7f75840c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f758a30b640 1 --2- 172.21.15.106:0/1922408904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75840c69f0 0x7f75840c6e00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7584070e40 tx=0x7f757c01c8c0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f758a30b640 1 -- 172.21.15.106:0/1922408904 shutdown_connections 2024-04-25T11:21:09.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f758a30b640 1 --2- 172.21.15.106:0/1922408904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75840c69f0 0x7f75840c6e00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f758a30b640 1 --2- 172.21.15.106:0/1922408904 >> v2:172.21.15.134:3300/0 conn(0x7f75840d0b10 0x7f75841622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f758a30b640 1 -- 172.21.15.106:0/1922408904 >> 172.21.15.106:0/1922408904 conn(0x7f7584068c40 msgr2=0x7f758406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f758a30b640 1 -- 172.21.15.106:0/1922408904 shutdown_connections 2024-04-25T11:21:09.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.418+0000 7f758a30b640 1 -- 172.21.15.106:0/1922408904 wait complete. 2024-04-25T11:21:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 Processor -- start 2024-04-25T11:21:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 -- start start 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75840c69f0 0x7f7584142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584154d40 0x7f75840cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75840cc030 0x7f7584157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f758406ed60 con 0x7f7584154d40 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f758406ebe0 con 0x7f75840c69f0 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f758416aa40 con 0x7f75840cc030 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584154d40 0x7f75840cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584154d40 0x7f75840cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36162/0 (socket says 172.21.15.106:36162) 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 -- 172.21.15.106:0/3691623591 learned_addr learned my addr 172.21.15.106:0/3691623591 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f7583fff640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75840c69f0 0x7f7584142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f75888a8640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75840cc030 0x7f7584157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 -- 172.21.15.106:0/3691623591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75840cc030 msgr2=0x7f7584157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75840cc030 0x7f7584157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 -- 172.21.15.106:0/3691623591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75840c69f0 msgr2=0x7f7584142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75840c69f0 0x7f7584142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 -- 172.21.15.106:0/3691623591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75840cc550 con 0x7f7584154d40 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f75888a8640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75840cc030 0x7f7584157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f757b7fe640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584154d40 0x7f75840cbaf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f756c00b3f0 tx=0x7f756c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c015020 con 0x7f7584154d40 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c011850 con 0x7f7584154d40 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75840c9130 con 0x7f7584154d40 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c011c40 con 0x7f7584154d40 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75840cac90 con 0x7f7584154d40 2024-04-25T11:21:09.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.422+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75840c97f0 con 0x7f7584154d40 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f756c018020 con 0x7f7584154d40 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75817fa640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756406f400 0x7f7564071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f756c08f580 con 0x7f7584154d40 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f7583fff640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756406f400 0x7f7564071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75817fa640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7f7564075310 0x7f7564077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 --> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7564077e30 con 0x7f7564075310 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f756c08fa10 con 0x7f7584154d40 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75888a8640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7f7564075310 0x7f7564077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f7583fff640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756406f400 0x7f7564071910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7574007fd0 tx=0x7f7574009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75888a8640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7f7564075310 0x7f7564077760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.426+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 <== osd.3 v2:172.21.15.106:6824/3914961038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7564077e30 con 0x7f7564075310 2024-04-25T11:21:09.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 --> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7584074520 con 0x7f7564075310 2024-04-25T11:21:09.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f75817fa640 1 -- 172.21.15.106:0/3691623591 <== osd.3 v2:172.21.15.106:6824/3914961038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7584074520 con 0x7f7564075310 2024-04-25T11:21:09.440 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:21:09.440 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:21:09.440 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7f7564075310 msgr2=0x7f7564077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6824/3914961038,v1:172.21.15.106:6825/3914961038] conn(0x7f7564075310 0x7f7564077760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756406f400 msgr2=0x7f7564071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f756406f400 0x7f7564071910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7574007fd0 tx=0x7f7574009040 comp rx=0 tx=0).stop 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584154d40 msgr2=0x7f75840cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 --2- 172.21.15.106:0/3691623591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584154d40 0x7f75840cbaf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f756c00b3f0 tx=0x7f756c00b8b0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f75888a8640 1 -- 172.21.15.106:0/3691623591 reap_dead start 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 shutdown_connections 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 >> 172.21.15.106:0/3691623591 conn(0x7f7584068c40 msgr2=0x7f758415f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 shutdown_connections 2024-04-25T11:21:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.438+0000 7f758a30b640 1 -- 172.21.15.106:0/3691623591 wait complete. 2024-04-25T11:21:09.457 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T11:21:09.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9e26c7640 1 Processor -- start 2024-04-25T11:21:09.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9e26c7640 1 -- start start 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9e26c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc160970 0x7ff9dc0d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9e26c7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc160ee0 con 0x7ff9dc1643d0 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9e26c7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc1611c0 con 0x7ff9dc0d26d0 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9e26c7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc0d06a0 con 0x7ff9dc160970 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9dbfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff9dc1643d0 0x7ff9dc0d1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45912/0 (socket says 172.21.15.106:45912) 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9dbfff640 1 -- 172.21.15.106:0/1142095509 learned_addr learned my addr 172.21.15.106:0/1142095509 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9e0c64640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc160970 0x7ff9dc0d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244008321 0 0) 0x7ff9dc160ee0 con 0x7ff9dc1643d0 2024-04-25T11:21:09.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.590+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9c8003530 con 0x7ff9dc1643d0 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1936685681 0 0) 0x7ff9c8003530 con 0x7ff9dc1643d0 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc160970 msgr2=0x7ff9dc0d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc160970 0x7ff9dc0d4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 >> v1:172.21.15.134:6789/0 conn(0x7ff9dc0d26d0 legacy=0x7ff9dc160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc1638f0 con 0x7ff9dc1643d0 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7ff9cc001a20 con 0x7ff9dc1643d0 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 >> v1:172.21.15.106:6789/0 conn(0x7ff9dc1643d0 legacy=0x7ff9dc0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9c8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc160970 0x7ff9c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8004d50 0x7ff9c8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0d06a0 con 0x7ff9c80039f0 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c800bb00 con 0x7ff9c8004d50 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c800bd00 con 0x7ff9dc160970 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9dbfff640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9c8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9db7fe640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc160970 0x7ff9c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9e0c64640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8004d50 0x7ff9c8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9dbfff640 1 -- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc160970 msgr2=0x7ff9c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9dbfff640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc160970 0x7ff9c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9dbfff640 1 -- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8004d50 msgr2=0x7ff9c8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9dbfff640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8004d50 0x7ff9c8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9dbfff640 1 -- 172.21.15.106:0/1142095509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c8003530 con 0x7ff9c80039f0 2024-04-25T11:21:09.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9dbfff640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9c8003e60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0d2d50 tx=0x7ff9cc01d260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710bd69c3c3fdaf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00a4a0 con 0x7ff9c80039f0 2024-04-25T11:21:09.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0ad650 con 0x7ff9c80039f0 2024-04-25T11:21:09.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.594+0000 7ff9daffd640 1 -- 172.21.15.106:0/1142095509 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0ada40 con 0x7ff9c80039f0 2024-04-25T11:21:09.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- 172.21.15.106:0/1142095509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 msgr2=0x7ff9c8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 --2- 172.21.15.106:0/1142095509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9c8003e60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0d2d50 tx=0x7ff9cc01d260 comp rx=0 tx=0).stop 2024-04-25T11:21:09.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e0c64640 1 -- 172.21.15.106:0/1142095509 reap_dead start 2024-04-25T11:21:09.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- 172.21.15.106:0/1142095509 shutdown_connections 2024-04-25T11:21:09.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- 172.21.15.106:0/1142095509 >> 172.21.15.106:0/1142095509 conn(0x7ff9dc06d830 msgr2=0x7ff9dc06dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- 172.21.15.106:0/1142095509 shutdown_connections 2024-04-25T11:21:09.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- 172.21.15.106:0/1142095509 wait complete. 2024-04-25T11:21:09.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 Processor -- start 2024-04-25T11:21:09.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- start start 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9dc0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b7e20 0x7ff9dc155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0b5230 0x7ff9dc15ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0d4ba0 con 0x7ff9c80039f0 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0d4a20 con 0x7ff9dc0b7e20 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e26c7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc0d4d20 con 0x7ff9dc0b5230 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9dc0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9dc0b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36188/0 (socket says 172.21.15.106:36188) 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 -- 172.21.15.106:0/3268069689 learned_addr learned my addr 172.21.15.106:0/3268069689 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9e0c64640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0b5230 0x7ff9dc15ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9db7fe640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b7e20 0x7ff9dc155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 -- 172.21.15.106:0/3268069689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0b5230 msgr2=0x7ff9dc15ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0b5230 0x7ff9dc15ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 -- 172.21.15.106:0/3268069689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b7e20 msgr2=0x7ff9dc155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b7e20 0x7ff9dc155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 -- 172.21.15.106:0/3268069689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc156330 con 0x7ff9c80039f0 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.598+0000 7ff9dbfff640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9dc0b6310 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc001d20 tx=0x7ff9c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9d8ff9640 1 -- 172.21.15.106:0/3268069689 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0bb070 con 0x7ff9c80039f0 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9d8ff9640 1 -- 172.21.15.106:0/3268069689 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0017e0 con 0x7ff9c80039f0 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc0b7060 con 0x7ff9c80039f0 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9d8ff9640 1 -- 172.21.15.106:0/3268069689 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc004b70 con 0x7ff9c80039f0 2024-04-25T11:21:09.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc0b7620 con 0x7ff9c80039f0 2024-04-25T11:21:09.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc050750 con 0x7ff9c80039f0 2024-04-25T11:21:09.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9d8ff9640 1 -- 172.21.15.106:0/3268069689 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff9cc0be070 con 0x7ff9c80039f0 2024-04-25T11:21:09.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9d8ff9640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff9bc06f400 0x7ff9bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9d8ff9640 1 -- 172.21.15.106:0/3268069689 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ff9cc129080 con 0x7ff9c80039f0 2024-04-25T11:21:09.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9db7fe640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff9bc06f400 0x7ff9bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.602+0000 7ff9db7fe640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff9bc06f400 0x7ff9bc071910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc160360 tx=0x7ff9c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.606+0000 7ff9d8ff9640 1 -- 172.21.15.106:0/3268069689 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9cc016f70 con 0x7ff9c80039f0 2024-04-25T11:21:09.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.754+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7ff9dc1532b0 con 0x7ff9bc06f400 2024-04-25T11:21:09.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.754+0000 7ff9d8ff9640 1 -- 172.21.15.106:0/3268069689 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff9c4007020 con 0x7ff9bc06f400 2024-04-25T11:21:09.755 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff9bc06f400 msgr2=0x7ff9bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff9bc06f400 0x7ff9bc071910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc160360 tx=0x7ff9c4009040 comp rx=0 tx=0).stop 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 msgr2=0x7ff9dc0b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9dc0b6310 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc001d20 tx=0x7ff9c8005840 comp rx=0 tx=0).stop 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 shutdown_connections 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9dc0b5230 0x7ff9dc15ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc0b7e20 0x7ff9dc155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7ff9bc06f400 0x7ff9bc071910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 --2- 172.21.15.106:0/3268069689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9c80039f0 0x7ff9dc0b6310 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 >> 172.21.15.106:0/3268069689 conn(0x7ff9dc06d830 msgr2=0x7ff9dc06dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 shutdown_connections 2024-04-25T11:21:09.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.758+0000 7ff9e26c7640 1 -- 172.21.15.106:0/3268069689 wait complete. 2024-04-25T11:21:09.775 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T11:21:09.775 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:21:09.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.922+0000 7fab9bb3e640 1 Processor -- start 2024-04-25T11:21:09.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.922+0000 7fab9bb3e640 1 -- start start 2024-04-25T11:21:09.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab9bb3e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab940b2f90 0x7fab940b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab9bb3e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fab9415e5c0 0x7fab9415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab9bb3e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab9415dc40 con 0x7fab9406ebe0 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab9bb3e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fab940b2df0 con 0x7fab940b2f90 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab9bb3e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fab940d0f60 con 0x7fab9415e5c0 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab998da640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab940b2f90 0x7fab940b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab998da640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab940b2f90 0x7fab940b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36202/0 (socket says 172.21.15.106:36202) 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab998da640 1 -- 172.21.15.106:0/3707139261 learned_addr learned my addr 172.21.15.106:0/3707139261 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab9a0db640 1 --2- 172.21.15.106:0/3707139261 >> v2:172.21.15.134:3300/0 conn(0x7fab9415e5c0 0x7fab9415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab998da640 1 -- 172.21.15.106:0/3707139261 >> v2:172.21.15.134:3300/0 conn(0x7fab9415e5c0 msgr2=0x7fab9415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab998da640 1 --2- 172.21.15.106:0/3707139261 >> v2:172.21.15.134:3300/0 conn(0x7fab9415e5c0 0x7fab9415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab998da640 1 -- 172.21.15.106:0/3707139261 >> v1:172.21.15.134:6789/0 conn(0x7fab9406ebe0 legacy=0x7fab9406eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab998da640 1 -- 172.21.15.106:0/3707139261 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab940d1240 con 0x7fab940b2f90 2024-04-25T11:21:09.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab998da640 1 --2- 172.21.15.106:0/3707139261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 0x7fab940b28b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fab9406fb80 tx=0x7fab8c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2a235b204874b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab988d8640 1 -- 172.21.15.106:0/3707139261 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0b2070 con 0x7fab940b2f90 2024-04-25T11:21:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab988d8640 1 -- 172.21.15.106:0/3707139261 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c0a9070 con 0x7fab940b2f90 2024-04-25T11:21:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.926+0000 7fab988d8640 1 -- 172.21.15.106:0/3707139261 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0ad880 con 0x7fab940b2f90 2024-04-25T11:21:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- 172.21.15.106:0/3707139261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 msgr2=0x7fab940b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 --2- 172.21.15.106:0/3707139261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 0x7fab940b28b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fab9406fb80 tx=0x7fab8c01c450 comp rx=0 tx=0).stop 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- 172.21.15.106:0/3707139261 shutdown_connections 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 --2- 172.21.15.106:0/3707139261 >> v2:172.21.15.134:3300/0 conn(0x7fab9415e5c0 0x7fab9415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 --2- 172.21.15.106:0/3707139261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 0x7fab940b28b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- 172.21.15.106:0/3707139261 >> 172.21.15.106:0/3707139261 conn(0x7fab94068c40 msgr2=0x7fab9406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- 172.21.15.106:0/3707139261 shutdown_connections 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- 172.21.15.106:0/3707139261 wait complete. 2024-04-25T11:21:09.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 Processor -- start 2024-04-25T11:21:09.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- start start 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab9406ebe0 0x7fab941401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 0x7fab941520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab9415e5c0 0x7fab94159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab9416a340 con 0x7fab940b2f90 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab9416a1c0 con 0x7fab9415e5c0 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9bb3e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab9416a4c0 con 0x7fab9406ebe0 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab990d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 0x7fab941520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab990d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 0x7fab941520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36214/0 (socket says 172.21.15.106:36214) 2024-04-25T11:21:09.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab990d9640 1 -- 172.21.15.106:0/849721795 learned_addr learned my addr 172.21.15.106:0/849721795 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab998da640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab9406ebe0 0x7fab941401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9a0db640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab9415e5c0 0x7fab94159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab990d9640 1 -- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab9406ebe0 msgr2=0x7fab941401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab990d9640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab9406ebe0 0x7fab941401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab990d9640 1 -- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab9415e5c0 msgr2=0x7fab94159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab990d9640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab9415e5c0 0x7fab94159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab990d9640 1 -- 172.21.15.106:0/849721795 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab941529b0 con 0x7fab940b2f90 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.930+0000 7fab9a0db640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab9415e5c0 0x7fab94159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab990d9640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 0x7fab941520d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fab8800b3e0 tx=0x7fab8800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88015020 con 0x7fab940b2f90 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab88011830 con 0x7fab940b2f90 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab94156370 con 0x7fab940b2f90 2024-04-25T11:21:09.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88011c40 con 0x7fab940b2f90 2024-04-25T11:21:09.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab94156930 con 0x7fab940b2f90 2024-04-25T11:21:09.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab94074520 con 0x7fab940b2f90 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fab88018020 con 0x7fab940b2f90 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fab8006f400 0x7fab80071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab998da640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fab8006f400 0x7fab80071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fab8808f580 con 0x7fab940b2f90 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fab80075310 0x7fab80077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 --> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab80077e30 con 0x7fab80075310 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fab8808fa10 con 0x7fab940b2f90 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab998da640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fab8006f400 0x7fab80071910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0af000 tx=0x7fab8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab9a0db640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fab80075310 0x7fab80077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.934+0000 7fab9a0db640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fab80075310 0x7fab80077760 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.938+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 <== osd.4 v2:172.21.15.134:6808/4246182191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab80077e30 con 0x7fab80075310 2024-04-25T11:21:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 --> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fab94050830 con 0x7fab80075310 2024-04-25T11:21:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab927fc640 1 -- 172.21.15.106:0/849721795 <== osd.4 v2:172.21.15.134:6808/4246182191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fab94050830 con 0x7fab80075310 2024-04-25T11:21:09.951 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:21:09.951 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:21:09.951 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:21:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fab80075310 msgr2=0x7fab80077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.134:6808/4246182191,v1:172.21.15.134:6809/4246182191] conn(0x7fab80075310 0x7fab80077760 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fab8006f400 msgr2=0x7fab80071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fab8006f400 0x7fab80071910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0af000 tx=0x7fab8c008040 comp rx=0 tx=0).stop 2024-04-25T11:21:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 msgr2=0x7fab941520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 --2- 172.21.15.106:0/849721795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab940b2f90 0x7fab941520d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fab8800b3e0 tx=0x7fab8800b8a0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9a0db640 1 -- 172.21.15.106:0/849721795 reap_dead start 2024-04-25T11:21:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 shutdown_connections 2024-04-25T11:21:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 >> 172.21.15.106:0/849721795 conn(0x7fab94068c40 msgr2=0x7fab94166f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 shutdown_connections 2024-04-25T11:21:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:09.950+0000 7fab9bb3e640 1 -- 172.21.15.106:0/849721795 wait complete. 2024-04-25T11:21:09.968 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T11:21:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f538895d640 1 Processor -- start 2024-04-25T11:21:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f538895d640 1 -- start start 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f538895d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f538406c820 0x7f538406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f538895d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f538406f7e0 0x7f538406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f538895d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f538406d170 con 0x7f5384073460 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f538895d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f538415f1a0 con 0x7f538406c820 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f538895d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5384072fd0 con 0x7f538406f7e0 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f5381d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f538406c820 0x7f538406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f5381d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f538406c820 0x7f538406cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36222/0 (socket says 172.21.15.106:36222) 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f5381d9b640 1 -- 172.21.15.106:0/1468540466 learned_addr learned my addr 172.21.15.106:0/1468540466 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f5382d9d640 1 --2- 172.21.15.106:0/1468540466 >> v2:172.21.15.134:3300/0 conn(0x7f538406f7e0 0x7f538406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f5381d9b640 1 -- 172.21.15.106:0/1468540466 >> v2:172.21.15.134:3300/0 conn(0x7f538406f7e0 msgr2=0x7f538406fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f5381d9b640 1 --2- 172.21.15.106:0/1468540466 >> v2:172.21.15.134:3300/0 conn(0x7f538406f7e0 0x7f538406fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f5381d9b640 1 -- 172.21.15.106:0/1468540466 >> v1:172.21.15.134:6789/0 conn(0x7f5384073460 legacy=0x7f538415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:10.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.102+0000 7f5381d9b640 1 -- 172.21.15.106:0/1468540466 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5384144350 con 0x7f538406c820 2024-04-25T11:21:10.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f5381d9b640 1 --2- 172.21.15.106:0/1468540466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406c820 0x7f538406cc30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5384072910 tx=0x7f536c01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6568bf840c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538159a640 1 -- 172.21.15.106:0/1468540466 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0b0070 con 0x7f538406c820 2024-04-25T11:21:10.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538159a640 1 -- 172.21.15.106:0/1468540466 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536c0ab600 con 0x7f538406c820 2024-04-25T11:21:10.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538159a640 1 -- 172.21.15.106:0/1468540466 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0ab9d0 con 0x7f538406c820 2024-04-25T11:21:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538895d640 1 -- 172.21.15.106:0/1468540466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406c820 msgr2=0x7f538406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538895d640 1 --2- 172.21.15.106:0/1468540466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406c820 0x7f538406cc30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5384072910 tx=0x7f536c01c2d0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538895d640 1 -- 172.21.15.106:0/1468540466 shutdown_connections 2024-04-25T11:21:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538895d640 1 --2- 172.21.15.106:0/1468540466 >> v2:172.21.15.134:3300/0 conn(0x7f538406f7e0 0x7f538406fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538895d640 1 --2- 172.21.15.106:0/1468540466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406c820 0x7f538406cc30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538895d640 1 -- 172.21.15.106:0/1468540466 >> 172.21.15.106:0/1468540466 conn(0x7f53840b3560 msgr2=0x7f53840b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538895d640 1 -- 172.21.15.106:0/1468540466 shutdown_connections 2024-04-25T11:21:10.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.106+0000 7f538895d640 1 -- 172.21.15.106:0/1468540466 wait complete. 2024-04-25T11:21:10.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 Processor -- start 2024-04-25T11:21:10.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 -- start start 2024-04-25T11:21:10.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f538406c820 0x7f538415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406f7e0 0x7f5384133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5384073460 0x7f5384139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5384073870 con 0x7f538406f7e0 2024-04-25T11:21:10.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f538415e8a0 con 0x7f538406c820 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53840739f0 con 0x7f5384073460 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406f7e0 0x7f5384133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406f7e0 0x7f5384133ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36236/0 (socket says 172.21.15.106:36236) 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 -- 172.21.15.106:0/4107585115 learned_addr learned my addr 172.21.15.106:0/4107585115 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538259c640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f538406c820 0x7f538415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5382d9d640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5384073460 0x7f5384139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 -- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5384073460 msgr2=0x7f5384139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5384073460 0x7f5384139510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 -- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f538406c820 msgr2=0x7f538415bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f538406c820 0x7f538415bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 -- 172.21.15.106:0/4107585115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538415c4e0 con 0x7f538406f7e0 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538259c640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f538406c820 0x7f538415bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f5381d9b640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406f7e0 0x7f5384133ef0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f536c01a980 tx=0x7f536c01ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f537affd640 1 -- 172.21.15.106:0/4107585115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0b0040 con 0x7f538406f7e0 2024-04-25T11:21:10.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f537affd640 1 -- 172.21.15.106:0/4107585115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536c0bc070 con 0x7f538406f7e0 2024-04-25T11:21:10.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538415ca70 con 0x7f538406f7e0 2024-04-25T11:21:10.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f537affd640 1 -- 172.21.15.106:0/4107585115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0b6660 con 0x7f538406f7e0 2024-04-25T11:21:10.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538415d030 con 0x7f538406f7e0 2024-04-25T11:21:10.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.110+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538406f390 con 0x7f538406f7e0 2024-04-25T11:21:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.114+0000 7f537affd640 1 -- 172.21.15.106:0/4107585115 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f536c0b6800 con 0x7f538406f7e0 2024-04-25T11:21:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.114+0000 7f537affd640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f536406f430 0x7f5364071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.114+0000 7f538259c640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f536406f430 0x7f5364071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.114+0000 7f537affd640 1 -- 172.21.15.106:0/4107585115 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f536c1292b0 con 0x7f538406f7e0 2024-04-25T11:21:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.114+0000 7f538259c640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f536406f430 0x7f5364071940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5384070ae0 tx=0x7f537c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.118+0000 7f537affd640 1 -- 172.21.15.106:0/4107585115 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f538406d170 con 0x7f538406f7e0 2024-04-25T11:21:10.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f538413c0f0 con 0x7f536406f430 2024-04-25T11:21:10.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f537affd640 1 -- 172.21.15.106:0/4107585115 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f538413c0f0 con 0x7f536406f430 2024-04-25T11:21:10.266 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T11:21:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f536406f430 msgr2=0x7f5364071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f536406f430 0x7f5364071940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5384070ae0 tx=0x7f537c008040 comp rx=0 tx=0).stop 2024-04-25T11:21:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406f7e0 msgr2=0x7f5384133ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406f7e0 0x7f5384133ef0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f536c01a980 tx=0x7f536c01ab10 comp rx=0 tx=0).stop 2024-04-25T11:21:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 shutdown_connections 2024-04-25T11:21:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5384073460 0x7f5384139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f536406f430 0x7f5364071940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f538406f7e0 0x7f5384133ef0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 --2- 172.21.15.106:0/4107585115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f538406c820 0x7f538415bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.266+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 >> 172.21.15.106:0/4107585115 conn(0x7f53840b3560 msgr2=0x7f5384144560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.270+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 shutdown_connections 2024-04-25T11:21:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.270+0000 7f538895d640 1 -- 172.21.15.106:0/4107585115 wait complete. 2024-04-25T11:21:10.287 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T11:21:10.287 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:21:10.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.422+0000 7f38d84c8640 1 Processor -- start 2024-04-25T11:21:10.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d84c8640 1 -- start start 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d84c8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38d00b2f90 0x7f38d00b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d84c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38d006ebe0 0x7f38d006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d84c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d006f530 con 0x7f38d0160610 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d84c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f38d00b2df0 con 0x7f38d006ebe0 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d84c8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f38d015dc40 con 0x7f38d00b2f90 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d5a63640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38d006ebe0 0x7f38d006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d6a65640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f38d0160610 0x7f38d0160a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45924/0 (socket says 172.21.15.106:45924) 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d6a65640 1 -- 172.21.15.106:0/1755070162 learned_addr learned my addr 172.21.15.106:0/1755070162 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d6264640 1 --2- 172.21.15.106:0/1755070162 >> v2:172.21.15.134:3300/0 conn(0x7f38d00b2f90 0x7f38d00b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d5a63640 1 -- 172.21.15.106:0/1755070162 >> v2:172.21.15.134:3300/0 conn(0x7f38d00b2f90 msgr2=0x7f38d00b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d5a63640 1 --2- 172.21.15.106:0/1755070162 >> v2:172.21.15.134:3300/0 conn(0x7f38d00b2f90 0x7f38d00b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d5a63640 1 -- 172.21.15.106:0/1755070162 >> v1:172.21.15.106:6789/0 conn(0x7f38d0160610 legacy=0x7f38d0160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.426+0000 7f38d5a63640 1 -- 172.21.15.106:0/1755070162 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d00d0f60 con 0x7f38d006ebe0 2024-04-25T11:21:10.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d5a63640 1 --2- 172.21.15.106:0/1755070162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 0x7f38d006eff0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f38d00723b0 tx=0x7f38c001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7391899ce27133 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d5262640 1 -- 172.21.15.106:0/1755070162 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00b0070 con 0x7f38d006ebe0 2024-04-25T11:21:10.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d5262640 1 -- 172.21.15.106:0/1755070162 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c00ab600 con 0x7f38d006ebe0 2024-04-25T11:21:10.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d5262640 1 -- 172.21.15.106:0/1755070162 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00ab9d0 con 0x7f38d006ebe0 2024-04-25T11:21:10.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 -- 172.21.15.106:0/1755070162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 msgr2=0x7f38d006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 --2- 172.21.15.106:0/1755070162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 0x7f38d006eff0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f38d00723b0 tx=0x7f38c001c3f0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 -- 172.21.15.106:0/1755070162 shutdown_connections 2024-04-25T11:21:10.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 --2- 172.21.15.106:0/1755070162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 0x7f38d006eff0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 --2- 172.21.15.106:0/1755070162 >> v2:172.21.15.134:3300/0 conn(0x7f38d00b2f90 0x7f38d00b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 -- 172.21.15.106:0/1755070162 >> 172.21.15.106:0/1755070162 conn(0x7f38d0068c40 msgr2=0x7f38d006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 -- 172.21.15.106:0/1755070162 shutdown_connections 2024-04-25T11:21:10.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 -- 172.21.15.106:0/1755070162 wait complete. 2024-04-25T11:21:10.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 Processor -- start 2024-04-25T11:21:10.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.430+0000 7f38d84c8640 1 -- start start 2024-04-25T11:21:10.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 0x7f38d01540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38d00b2f90 0x7f38d0158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38d0160610 0x7f38d00b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0168330 con 0x7f38d006ebe0 2024-04-25T11:21:10.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38d01681b0 con 0x7f38d00b2f90 2024-04-25T11:21:10.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38d01684b0 con 0x7f38d0160610 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 0x7f38d01540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 0x7f38d01540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36256/0 (socket says 172.21.15.106:36256) 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 -- 172.21.15.106:0/3552849886 learned_addr learned my addr 172.21.15.106:0/3552849886 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d5a63640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38d00b2f90 0x7f38d0158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 -- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38d0160610 msgr2=0x7f38d00b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38d0160610 0x7f38d00b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 -- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38d00b2f90 msgr2=0x7f38d0158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38d00b2f90 0x7f38d0158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 -- 172.21.15.106:0/3552849886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d015a140 con 0x7f38d006ebe0 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d5a63640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38d00b2f90 0x7f38d0158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d6264640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 0x7f38d01540f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00d5f0 tx=0x7f38cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc004240 con 0x7f38d006ebe0 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38cc0043e0 con 0x7f38d006ebe0 2024-04-25T11:21:10.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc003840 con 0x7f38d006ebe0 2024-04-25T11:21:10.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d0159b80 con 0x7f38d006ebe0 2024-04-25T11:21:10.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d01f7d10 con 0x7f38d006ebe0 2024-04-25T11:21:10.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38d0074520 con 0x7f38d006ebe0 2024-04-25T11:21:10.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f38cc0039e0 con 0x7f38d006ebe0 2024-04-25T11:21:10.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38b406f400 0x7f38b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d5a63640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38b406f400 0x7f38b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f38cc08fd40 con 0x7f38d006ebe0 2024-04-25T11:21:10.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f38b4075310 0x7f38b4077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 --> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38cc001d40 con 0x7f38b4075310 2024-04-25T11:21:10.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f38cc0901d0 con 0x7f38d006ebe0 2024-04-25T11:21:10.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.434+0000 7f38d5a63640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38b406f400 0x7f38b4071910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f38c00ad000 tx=0x7f38c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.438+0000 7f38d6a65640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f38b4075310 0x7f38b4077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.438+0000 7f38d6a65640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f38b4075310 0x7f38b4077760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.438+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 <== osd.5 v2:172.21.15.134:6800/1995022176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f38cc001d40 con 0x7f38b4075310 2024-04-25T11:21:10.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 --> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f38d01f7af0 con 0x7f38b4075310 2024-04-25T11:21:10.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38c6ffd640 1 -- 172.21.15.106:0/3552849886 <== osd.5 v2:172.21.15.134:6800/1995022176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f38d01f7af0 con 0x7f38b4075310 2024-04-25T11:21:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:21:10.452 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:21:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:21:10.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f38b4075310 msgr2=0x7f38b4077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38d84c8640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.134:6800/1995022176,v1:172.21.15.134:6801/1995022176] conn(0x7f38b4075310 0x7f38b4077760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38b406f400 msgr2=0x7f38b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38d84c8640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f38b406f400 0x7f38b4071910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f38c00ad000 tx=0x7f38c0008040 comp rx=0 tx=0).stop 2024-04-25T11:21:10.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 msgr2=0x7f38d01540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38d84c8640 1 --2- 172.21.15.106:0/3552849886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38d006ebe0 0x7f38d01540f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00d5f0 tx=0x7f38cc00dab0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.450+0000 7f38d6a65640 1 -- 172.21.15.106:0/3552849886 reap_dead start 2024-04-25T11:21:10.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.454+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 shutdown_connections 2024-04-25T11:21:10.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.454+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 >> 172.21.15.106:0/3552849886 conn(0x7f38d0068c40 msgr2=0x7f38d00cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.454+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 shutdown_connections 2024-04-25T11:21:10.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.454+0000 7f38d84c8640 1 -- 172.21.15.106:0/3552849886 wait complete. 2024-04-25T11:21:10.470 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T11:21:10.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac3a79640 1 Processor -- start 2024-04-25T11:21:10.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac3a79640 1 -- start start 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac3a79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5abc160fc0 0x7f5abc0b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac3a79640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5abc0b5110 0x7f5abc06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac3a79640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5abc0b5520 con 0x7f5abc15cae0 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac3a79640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5abc0ce2a0 con 0x7f5abc160fc0 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac3a79640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5abc0b4460 con 0x7f5abc0b5110 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac1014640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5abc160fc0 0x7f5abc0b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac1014640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5abc160fc0 0x7f5abc0b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36258/0 (socket says 172.21.15.106:36258) 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac1014640 1 -- 172.21.15.106:0/2757370632 learned_addr learned my addr 172.21.15.106:0/2757370632 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac1815640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5abc15cae0 0x7f5abc0cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45928/0 (socket says 172.21.15.106:45928) 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac2016640 1 --2- 172.21.15.106:0/2757370632 >> v2:172.21.15.134:3300/0 conn(0x7f5abc0b5110 0x7f5abc06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac1014640 1 -- 172.21.15.106:0/2757370632 >> v2:172.21.15.134:3300/0 conn(0x7f5abc0b5110 msgr2=0x7f5abc06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:10.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac1014640 1 --2- 172.21.15.106:0/2757370632 >> v2:172.21.15.134:3300/0 conn(0x7f5abc0b5110 0x7f5abc06c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac1014640 1 -- 172.21.15.106:0/2757370632 >> v1:172.21.15.106:6789/0 conn(0x7f5abc15cae0 legacy=0x7f5abc0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac0813640 1 -- 172.21.15.106:0/2757370632 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1098624396 0 0) 0x7f5ab4001120 con 0x7f5abc15cae0 2024-04-25T11:21:10.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.614+0000 7f5ac1014640 1 -- 172.21.15.106:0/2757370632 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5abc1613d0 con 0x7f5abc160fc0 2024-04-25T11:21:10.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac1014640 1 --2- 172.21.15.106:0/2757370632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc160fc0 0x7f5abc0b4bd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5abc1616b0 tx=0x7f5ab001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715c1b08e6772820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac0813640 1 -- 172.21.15.106:0/2757370632 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00b4070 con 0x7f5abc160fc0 2024-04-25T11:21:10.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac0813640 1 -- 172.21.15.106:0/2757370632 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab00a7930 con 0x7f5abc160fc0 2024-04-25T11:21:10.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac0813640 1 -- 172.21.15.106:0/2757370632 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00a7d00 con 0x7f5abc160fc0 2024-04-25T11:21:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2757370632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc160fc0 msgr2=0x7f5abc0b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2757370632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc160fc0 0x7f5abc0b4bd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5abc1616b0 tx=0x7f5ab001eb60 comp rx=0 tx=0).stop 2024-04-25T11:21:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2757370632 shutdown_connections 2024-04-25T11:21:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2757370632 >> v2:172.21.15.134:3300/0 conn(0x7f5abc0b5110 0x7f5abc06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2757370632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc160fc0 0x7f5abc0b4bd0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2757370632 >> 172.21.15.106:0/2757370632 conn(0x7f5abc0b3560 msgr2=0x7f5abc0b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2757370632 shutdown_connections 2024-04-25T11:21:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.618+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2757370632 wait complete. 2024-04-25T11:21:10.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 Processor -- start 2024-04-25T11:21:10.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 -- start start 2024-04-25T11:21:10.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc0b5110 0x7f5abc151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5abc15cae0 0x7f5abc1472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc160fc0 0x7f5abc154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc06d170 con 0x7f5abc0b5110 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc0b4740 con 0x7f5abc160fc0 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5abc0cfde0 con 0x7f5abc15cae0 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc0b5110 0x7f5abc151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc0b5110 0x7f5abc151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36274/0 (socket says 172.21.15.106:36274) 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 -- 172.21.15.106:0/2630684405 learned_addr learned my addr 172.21.15.106:0/2630684405 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1014640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5abc15cae0 0x7f5abc1472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac2016640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc160fc0 0x7f5abc154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 -- 172.21.15.106:0/2630684405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5abc15cae0 msgr2=0x7f5abc1472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5abc15cae0 0x7f5abc1472b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 -- 172.21.15.106:0/2630684405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc160fc0 msgr2=0x7f5abc154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc160fc0 0x7f5abc154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 -- 172.21.15.106:0/2630684405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5abc0b0be0 con 0x7f5abc0b5110 2024-04-25T11:21:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac1815640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc0b5110 0x7f5abc151c20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400c480 tx=0x7f5ab400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5aba7fc640 1 -- 172.21.15.106:0/2630684405 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4018070 con 0x7f5abc0b5110 2024-04-25T11:21:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5aba7fc640 1 -- 172.21.15.106:0/2630684405 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab4007500 con 0x7f5abc0b5110 2024-04-25T11:21:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5abc152250 con 0x7f5abc0b5110 2024-04-25T11:21:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5aba7fc640 1 -- 172.21.15.106:0/2630684405 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4004a30 con 0x7f5abc0b5110 2024-04-25T11:21:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5abc148900 con 0x7f5abc0b5110 2024-04-25T11:21:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.622+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5abc050750 con 0x7f5abc0b5110 2024-04-25T11:21:10.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.626+0000 7f5aba7fc640 1 -- 172.21.15.106:0/2630684405 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ab401b070 con 0x7f5abc0b5110 2024-04-25T11:21:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.626+0000 7f5aba7fc640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa806f400 0x7f5aa8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.626+0000 7f5aba7fc640 1 -- 172.21.15.106:0/2630684405 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f5ab408e4d0 con 0x7f5abc0b5110 2024-04-25T11:21:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.626+0000 7f5ac1014640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa806f400 0x7f5aa8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.626+0000 7f5ac1014640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa806f400 0x7f5aa8071910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00b1000 tx=0x7f5ab00a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.630+0000 7f5aba7fc640 1 -- 172.21.15.106:0/2630684405 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5abc0b5520 con 0x7f5abc0b5110 2024-04-25T11:21:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.782+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f5abc0b6780 con 0x7f5aa806f400 2024-04-25T11:21:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5aba7fc640 1 -- 172.21.15.106:0/2630684405 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5abc0b6780 con 0x7f5aa806f400 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa806f400 msgr2=0x7f5aa8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa806f400 0x7f5aa8071910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00b1000 tx=0x7f5ab00a9070 comp rx=0 tx=0).stop 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc0b5110 msgr2=0x7f5abc151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc0b5110 0x7f5abc151c20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400c480 tx=0x7f5ab400c940 comp rx=0 tx=0).stop 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 shutdown_connections 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc160fc0 0x7f5abc154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5abc15cae0 0x7f5abc1472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f5aa806f400 0x7f5aa8071910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 --2- 172.21.15.106:0/2630684405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5abc0b5110 0x7f5abc151c20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 >> 172.21.15.106:0/2630684405 conn(0x7f5abc0b3560 msgr2=0x7f5abc0d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 shutdown_connections 2024-04-25T11:21:10.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.786+0000 7f5ac3a79640 1 -- 172.21.15.106:0/2630684405 wait complete. 2024-04-25T11:21:10.805 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T11:21:10.805 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:21:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c8fc6640 1 Processor -- start 2024-04-25T11:21:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c8fc6640 1 -- start start 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c8fc6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4160400 0x7f58c415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c8fc6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58c40b0790 0x7f58c4072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c8fc6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c40b12a0 con 0x7f58c4076230 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c8fc6640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f58c40b0fc0 con 0x7f58c40b0790 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c8fc6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f58c415f920 con 0x7f58c4160400 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c1d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58c40b0790 0x7f58c4072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f58c4076230 0x7f58c406ef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45944/0 (socket says 172.21.15.106:45944) 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c1d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58c40b0790 0x7f58c4072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36282/0 (socket says 172.21.15.106:36282) 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c2d9d640 1 -- 172.21.15.106:0/2236333769 learned_addr learned my addr 172.21.15.106:0/2236333769 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c259c640 1 --2- 172.21.15.106:0/2236333769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4160400 0x7f58c415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c1d9b640 1 -- 172.21.15.106:0/2236333769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4160400 msgr2=0x7f58c415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c1d9b640 1 --2- 172.21.15.106:0/2236333769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4160400 0x7f58c415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c1d9b640 1 -- 172.21.15.106:0/2236333769 >> v1:172.21.15.106:6789/0 conn(0x7f58c4076230 legacy=0x7f58c406ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.958+0000 7f58c1d9b640 1 -- 172.21.15.106:0/2236333769 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c415fc00 con 0x7f58c40b0790 2024-04-25T11:21:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c1d9b640 1 --2- 172.21.15.106:0/2236333769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 0x7f58c4072f00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009d90 tx=0x7f58b801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecd234e605fcea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c159a640 1 -- 172.21.15.106:0/2236333769 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b80b2070 con 0x7f58c40b0790 2024-04-25T11:21:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c159a640 1 -- 172.21.15.106:0/2236333769 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b80a5830 con 0x7f58c40b0790 2024-04-25T11:21:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c159a640 1 -- 172.21.15.106:0/2236333769 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b80a5c00 con 0x7f58c40b0790 2024-04-25T11:21:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c8fc6640 1 -- 172.21.15.106:0/2236333769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 msgr2=0x7f58c4072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c8fc6640 1 --2- 172.21.15.106:0/2236333769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 0x7f58c4072f00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009d90 tx=0x7f58b801c970 comp rx=0 tx=0).stop 2024-04-25T11:21:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c8fc6640 1 -- 172.21.15.106:0/2236333769 shutdown_connections 2024-04-25T11:21:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c8fc6640 1 --2- 172.21.15.106:0/2236333769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 0x7f58c4072f00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c8fc6640 1 --2- 172.21.15.106:0/2236333769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4160400 0x7f58c415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c8fc6640 1 -- 172.21.15.106:0/2236333769 >> 172.21.15.106:0/2236333769 conn(0x7f58c40b3550 msgr2=0x7f58c40b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c8fc6640 1 -- 172.21.15.106:0/2236333769 shutdown_connections 2024-04-25T11:21:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.962+0000 7f58c8fc6640 1 -- 172.21.15.106:0/2236333769 wait complete. 2024-04-25T11:21:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 Processor -- start 2024-04-25T11:21:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 -- start start 2024-04-25T11:21:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4076230 0x7f58c415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 0x7f58c4143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c4160400 0x7f58c40abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58c40704b0 con 0x7f58c40b0790 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4070330 con 0x7f58c4160400 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f58c4070630 con 0x7f58c4076230 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 0x7f58c4143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 0x7f58c4143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36288/0 (socket says 172.21.15.106:36288) 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 -- 172.21.15.106:0/3756420213 learned_addr learned my addr 172.21.15.106:0/3756420213 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c2d9d640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c4160400 0x7f58c40abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c259c640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4076230 0x7f58c415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 -- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4076230 msgr2=0x7f58c415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58c4076230 0x7f58c415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 -- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c4160400 msgr2=0x7f58c40abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c4160400 0x7f58c40abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 -- 172.21.15.106:0/3756420213 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c4151e70 con 0x7f58c40b0790 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c1d9b640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 0x7f58c4143150 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f58b801a5a0 tx=0x7f58b8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b80b2050 con 0x7f58c40b0790 2024-04-25T11:21:10.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b80be070 con 0x7f58c40b0790 2024-04-25T11:21:10.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c4152150 con 0x7f58c40b0790 2024-04-25T11:21:10.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b80ada70 con 0x7f58c40b0790 2024-04-25T11:21:10.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c4159ef0 con 0x7f58c40b0790 2024-04-25T11:21:10.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.966+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58c4152670 con 0x7f58c40b0790 2024-04-25T11:21:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f58b80adc10 con 0x7f58c40b0790 2024-04-25T11:21:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58b2ffd640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f58a406f400 0x7f58a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f58b812ab70 con 0x7f58c40b0790 2024-04-25T11:21:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58c259c640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f58a406f400 0x7f58a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58b2ffd640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7f58a4075310 0x7f58a4077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 --> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58a4077e30 con 0x7f58a4075310 2024-04-25T11:21:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f58b812b000 con 0x7f58c40b0790 2024-04-25T11:21:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58c259c640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f58a406f400 0x7f58a4071910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f58c40b6a40 tx=0x7f58bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58c2d9d640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7f58a4075310 0x7f58a4077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.970+0000 7f58c2d9d640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7f58a4075310 0x7f58a4077760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.974+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 <== osd.6 v2:172.21.15.134:6824/2059200670 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58a4077e30 con 0x7f58a4075310 2024-04-25T11:21:10.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 --> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f58c415b3f0 con 0x7f58a4075310 2024-04-25T11:21:10.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58b2ffd640 1 -- 172.21.15.106:0/3756420213 <== osd.6 v2:172.21.15.134:6824/2059200670 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f58c415b3f0 con 0x7f58a4075310 2024-04-25T11:21:10.986 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:21:10.987 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:21:10.987 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:21:10.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7f58a4075310 msgr2=0x7f58a4077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.134:6824/2059200670,v1:172.21.15.134:6825/2059200670] conn(0x7f58a4075310 0x7f58a4077760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f58a406f400 msgr2=0x7f58a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f58a406f400 0x7f58a4071910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f58c40b6a40 tx=0x7f58bc008040 comp rx=0 tx=0).stop 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 msgr2=0x7f58c4143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 --2- 172.21.15.106:0/3756420213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c40b0790 0x7f58c4143150 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f58b801a5a0 tx=0x7f58b8019710 comp rx=0 tx=0).stop 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c2d9d640 1 -- 172.21.15.106:0/3756420213 reap_dead start 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 shutdown_connections 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 >> 172.21.15.106:0/3756420213 conn(0x7f58c40b3550 msgr2=0x7f58c40b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 shutdown_connections 2024-04-25T11:21:10.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:10.986+0000 7f58c8fc6640 1 -- 172.21.15.106:0/3756420213 wait complete. 2024-04-25T11:21:11.005 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T11:21:11.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f470af1c640 1 Processor -- start 2024-04-25T11:21:11.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f470af1c640 1 -- start start 2024-04-25T11:21:11.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f470af1c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406d110 0x7f47040b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f470af1c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47040b75d0 con 0x7f470406cd00 2024-04-25T11:21:11.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f470af1c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47040b7870 con 0x7f47040b39e0 2024-04-25T11:21:11.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f470af1c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f47040b1dd0 con 0x7f470406d110 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f4703fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f470406cd00 0x7f47040b2260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45952/0 (socket says 172.21.15.106:45952) 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f4703fff640 1 -- 172.21.15.106:0/801878845 learned_addr learned my addr 172.21.15.106:0/801878845 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f47094b9640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406d110 0x7f47040b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099230763 0 0) 0x7f47040b75d0 con 0x7f470406cd00 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.146+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46ec003530 con 0x7f470406cd00 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4069232818 0 0) 0x7f46ec003530 con 0x7f470406cd00 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406d110 msgr2=0x7f47040b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406d110 0x7f47040b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 >> v1:172.21.15.134:6789/0 conn(0x7f47040b39e0 legacy=0x7f470406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47040b1a60 con 0x7f470406cd00 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f46f8001680 con 0x7f470406cd00 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 >> v1:172.21.15.106:6789/0 conn(0x7f470406cd00 legacy=0x7f47040b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406d110 0x7f46ec0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec003f70 0x7f46ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46ec004d50 0x7f46ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f47040b1dd0 con 0x7f46ec004d50 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec00bc90 con 0x7f46ec003f70 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec00be90 con 0x7f470406d110 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47094b9640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46ec004d50 0x7f46ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f4708cb8640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406d110 0x7f46ec0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f4703fff640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec003f70 0x7f46ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47094b9640 1 -- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406d110 msgr2=0x7f46ec0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47094b9640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406d110 0x7f46ec0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47094b9640 1 -- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec003f70 msgr2=0x7f46ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47094b9640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec003f70 0x7f46ec004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47094b9640 1 -- 172.21.15.106:0/801878845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec003530 con 0x7f46ec004d50 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47094b9640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46ec004d50 0x7f46ec005200 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4704072620 tx=0x7f46fc01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba5e9f7e4b0ee4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0b2070 con 0x7f46ec004d50 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc0a5b40 con 0x7f46ec004d50 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47037fe640 1 -- 172.21.15.106:0/801878845 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc009c10 con 0x7f46ec004d50 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f470af1c640 1 -- 172.21.15.106:0/801878845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46ec004d50 msgr2=0x7f46ec005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f470af1c640 1 --2- 172.21.15.106:0/801878845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46ec004d50 0x7f46ec005200 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4704072620 tx=0x7f46fc01cc40 comp rx=0 tx=0).stop 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f47094b9640 1 -- 172.21.15.106:0/801878845 reap_dead start 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f470af1c640 1 -- 172.21.15.106:0/801878845 shutdown_connections 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f470af1c640 1 -- 172.21.15.106:0/801878845 >> 172.21.15.106:0/801878845 conn(0x7f470409ba30 msgr2=0x7f47040b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.150+0000 7f470af1c640 1 -- 172.21.15.106:0/801878845 shutdown_connections 2024-04-25T11:21:11.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 -- 172.21.15.106:0/801878845 wait complete. 2024-04-25T11:21:11.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 Processor -- start 2024-04-25T11:21:11.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 -- start start 2024-04-25T11:21:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4704158240 0x7f470409d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4704157ba0 0x7f4704169010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4704169550 0x7f47041fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f470416bd80 con 0x7f4704157ba0 2024-04-25T11:21:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47040b3390 con 0x7f4704158240 2024-04-25T11:21:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f470af1c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f470416bf00 con 0x7f4704169550 2024-04-25T11:21:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4704157ba0 0x7f4704169010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4704157ba0 0x7f4704169010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36294/0 (socket says 172.21.15.106:36294) 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 -- 172.21.15.106:0/4132567228 learned_addr learned my addr 172.21.15.106:0/4132567228 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4708cb8640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4704158240 0x7f470409d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f47094b9640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4704169550 0x7f47041fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 -- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4704169550 msgr2=0x7f47041fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4704169550 0x7f47041fcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 -- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4704158240 msgr2=0x7f470409d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4704158240 0x7f470409d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 -- 172.21.15.106:0/4132567228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4704169960 con 0x7f4704157ba0 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4708cb8640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4704158240 0x7f470409d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.154+0000 7f4703fff640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4704157ba0 0x7f4704169010 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f46f800d780 tx=0x7f46f800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f47017fa640 1 -- 172.21.15.106:0/4132567228 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8019070 con 0x7f4704157ba0 2024-04-25T11:21:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f47017fa640 1 -- 172.21.15.106:0/4132567228 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f80074e0 con 0x7f4704157ba0 2024-04-25T11:21:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470409dd50 con 0x7f4704157ba0 2024-04-25T11:21:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f47017fa640 1 -- 172.21.15.106:0/4132567228 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004a30 con 0x7f4704157ba0 2024-04-25T11:21:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470409e310 con 0x7f4704157ba0 2024-04-25T11:21:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4704050750 con 0x7f4704157ba0 2024-04-25T11:21:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f47017fa640 1 -- 172.21.15.106:0/4132567228 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f46f801c070 con 0x7f4704157ba0 2024-04-25T11:21:11.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f47017fa640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f46f006f400 0x7f46f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f47017fa640 1 -- 172.21.15.106:0/4132567228 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f46f808e9b0 con 0x7f4704157ba0 2024-04-25T11:21:11.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f4708cb8640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f46f006f400 0x7f46f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.158+0000 7f4708cb8640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f46f006f400 0x7f46f0071910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f470409f270 tx=0x7f46f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.162+0000 7f47017fa640 1 -- 172.21.15.106:0/4132567228 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4704169960 con 0x7f4704157ba0 2024-04-25T11:21:11.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.310+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f4704074510 con 0x7f46f006f400 2024-04-25T11:21:11.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.310+0000 7f47017fa640 1 -- 172.21.15.106:0/4132567228 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4704074510 con 0x7f46f006f400 2024-04-25T11:21:11.311 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T11:21:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f46f006f400 msgr2=0x7f46f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f46f006f400 0x7f46f0071910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f470409f270 tx=0x7f46f4009040 comp rx=0 tx=0).stop 2024-04-25T11:21:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4704157ba0 msgr2=0x7f4704169010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4704157ba0 0x7f4704169010 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f46f800d780 tx=0x7f46f800dc40 comp rx=0 tx=0).stop 2024-04-25T11:21:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 shutdown_connections 2024-04-25T11:21:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4704169550 0x7f47041fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f46f006f400 0x7f46f0071910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4704157ba0 0x7f4704169010 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 --2- 172.21.15.106:0/4132567228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4704158240 0x7f470409d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 >> 172.21.15.106:0/4132567228 conn(0x7f470409ba30 msgr2=0x7f470406fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 shutdown_connections 2024-04-25T11:21:11.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.314+0000 7f470af1c640 1 -- 172.21.15.106:0/4132567228 wait complete. 2024-04-25T11:21:11.331 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T11:21:11.331 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:21:11.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b61246640 1 Processor -- start 2024-04-25T11:21:11.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b61246640 1 -- start start 2024-04-25T11:21:11.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b61246640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7b5c154d40 0x7f7b5c1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b61246640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b5c0d0400 0x7f7b5c0d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b61246640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7b5c072be0 0x7f7b5c072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b61246640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7b5c0d0d50 con 0x7f7b5c072be0 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b61246640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7b5c155150 con 0x7f7b5c154d40 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b61246640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5c15e610 con 0x7f7b5c0d0400 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b5b59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7b5c072be0 0x7f7b5c072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b5b59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7b5c072be0 0x7f7b5c072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36296/0 (socket says 172.21.15.106:36296) 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b5b59e640 1 -- 172.21.15.106:0/2874463421 learned_addr learned my addr 172.21.15.106:0/2874463421 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.486+0000 7f7b5ad9d640 1 --2- 172.21.15.106:0/2874463421 >> v2:172.21.15.134:3300/0 conn(0x7f7b5c154d40 0x7f7b5c1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b5b59e640 1 -- 172.21.15.106:0/2874463421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b5c0d0400 msgr2=0x7f7b5c0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b5b59e640 1 --2- 172.21.15.106:0/2874463421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b5c0d0400 0x7f7b5c0d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b5b59e640 1 -- 172.21.15.106:0/2874463421 >> v2:172.21.15.134:3300/0 conn(0x7f7b5c154d40 msgr2=0x7f7b5c1622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b5b59e640 1 --2- 172.21.15.106:0/2874463421 >> v2:172.21.15.134:3300/0 conn(0x7f7b5c154d40 0x7f7b5c1622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b5b59e640 1 -- 172.21.15.106:0/2874463421 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c15e8f0 con 0x7f7b5c072be0 2024-04-25T11:21:11.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b5ad9d640 1 --2- 172.21.15.106:0/2874463421 >> v2:172.21.15.134:3300/0 conn(0x7f7b5c154d40 0x7f7b5c1622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:11.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b5b59e640 1 --2- 172.21.15.106:0/2874463421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c072be0 0x7f7b5c072ff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c071050 tx=0x7f7b5401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501966e0a6fdf727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b59d9b640 1 -- 172.21.15.106:0/2874463421 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0c7a00 con 0x7f7b5c072be0 2024-04-25T11:21:11.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b59d9b640 1 -- 172.21.15.106:0/2874463421 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b540a56d0 con 0x7f7b5c072be0 2024-04-25T11:21:11.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b59d9b640 1 -- 172.21.15.106:0/2874463421 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b540a5aa0 con 0x7f7b5c072be0 2024-04-25T11:21:11.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 -- 172.21.15.106:0/2874463421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c072be0 msgr2=0x7f7b5c072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 --2- 172.21.15.106:0/2874463421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c072be0 0x7f7b5c072ff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c071050 tx=0x7f7b5401c960 comp rx=0 tx=0).stop 2024-04-25T11:21:11.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 -- 172.21.15.106:0/2874463421 shutdown_connections 2024-04-25T11:21:11.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 --2- 172.21.15.106:0/2874463421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c072be0 0x7f7b5c072ff0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 --2- 172.21.15.106:0/2874463421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b5c0d0400 0x7f7b5c0d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 --2- 172.21.15.106:0/2874463421 >> v2:172.21.15.134:3300/0 conn(0x7f7b5c154d40 0x7f7b5c1622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 -- 172.21.15.106:0/2874463421 >> 172.21.15.106:0/2874463421 conn(0x7f7b5c068c40 msgr2=0x7f7b5c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 -- 172.21.15.106:0/2874463421 shutdown_connections 2024-04-25T11:21:11.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.490+0000 7f7b61246640 1 -- 172.21.15.106:0/2874463421 wait complete. 2024-04-25T11:21:11.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 Processor -- start 2024-04-25T11:21:11.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 -- start start 2024-04-25T11:21:11.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b5c072be0 0x7f7b5c1fa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c0d0400 0x7f7b5c15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b5c154d40 0x7f7b5c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c073940 con 0x7f7b5c0d0400 2024-04-25T11:21:11.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c15e2c0 con 0x7f7b5c072be0 2024-04-25T11:21:11.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5c06ebe0 con 0x7f7b5c154d40 2024-04-25T11:21:11.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c0d0400 0x7f7b5c15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c0d0400 0x7f7b5c15ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36306/0 (socket says 172.21.15.106:36306) 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 -- 172.21.15.106:0/2877888270 learned_addr learned my addr 172.21.15.106:0/2877888270 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5ad9d640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b5c072be0 0x7f7b5c1fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5b59e640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b5c154d40 0x7f7b5c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 -- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b5c154d40 msgr2=0x7f7b5c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b5c154d40 0x7f7b5c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 -- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b5c072be0 msgr2=0x7f7b5c1fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b5c072be0 0x7f7b5c1fa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 -- 172.21.15.106:0/2877888270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c1fa570 con 0x7f7b5c0d0400 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b5a59c640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c0d0400 0x7f7b5c15ccd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000b420 tx=0x7f7b5000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50011830 con 0x7f7b5c0d0400 2024-04-25T11:21:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b50016070 con 0x7f7b5c0d0400 2024-04-25T11:21:11.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c0d01b0 con 0x7f7b5c0d0400 2024-04-25T11:21:11.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50011c10 con 0x7f7b5c0d0400 2024-04-25T11:21:11.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.494+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c14d490 con 0x7f7b5c0d0400 2024-04-25T11:21:11.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b5c074520 con 0x7f7b5c0d0400 2024-04-25T11:21:11.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7b5001b460 con 0x7f7b5c0d0400 2024-04-25T11:21:11.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b437fe640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7b4806f400 0x7f7b48071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b5ad9d640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7b4806f400 0x7f7b48071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f7b5008f660 con 0x7f7b5c0d0400 2024-04-25T11:21:11.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b437fe640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f7b48075310 0x7f7b48077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 --> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b48077e30 con 0x7f7b48075310 2024-04-25T11:21:11.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7b5008faf0 con 0x7f7b5c0d0400 2024-04-25T11:21:11.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b5ad9d640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7b4806f400 0x7f7b48071910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c006fd0 tx=0x7f7b4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b5b59e640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f7b48075310 0x7f7b48077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.498+0000 7f7b5b59e640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f7b48075310 0x7f7b48077760 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.502+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 <== osd.7 v2:172.21.15.134:6816/1198352866 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b48077e30 con 0x7f7b48075310 2024-04-25T11:21:11.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 --> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7b5c0cff60 con 0x7f7b48075310 2024-04-25T11:21:11.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b437fe640 1 -- 172.21.15.106:0/2877888270 <== osd.7 v2:172.21.15.134:6816/1198352866 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7b5c0cff60 con 0x7f7b48075310 2024-04-25T11:21:11.515 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T11:21:11.515 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:21:11.516 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T11:21:11.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f7b48075310 msgr2=0x7f7b48077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.134:6816/1198352866,v1:172.21.15.134:6817/1198352866] conn(0x7f7b48075310 0x7f7b48077760 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7b4806f400 msgr2=0x7f7b48071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f7b4806f400 0x7f7b48071910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c006fd0 tx=0x7f7b4c008040 comp rx=0 tx=0).stop 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c0d0400 msgr2=0x7f7b5c15ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 --2- 172.21.15.106:0/2877888270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b5c0d0400 0x7f7b5c15ccd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000b420 tx=0x7f7b5000b8e0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b5b59e640 1 -- 172.21.15.106:0/2877888270 reap_dead start 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 shutdown_connections 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 >> 172.21.15.106:0/2877888270 conn(0x7f7b5c068c40 msgr2=0x7f7b5c153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 shutdown_connections 2024-04-25T11:21:11.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.514+0000 7f7b61246640 1 -- 172.21.15.106:0/2877888270 wait complete. 2024-04-25T11:21:11.535 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T11:21:11.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77d191e640 1 Processor -- start 2024-04-25T11:21:11.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77d191e640 1 -- start start 2024-04-25T11:21:11.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77d191e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc160fc0 0x7f77cc0b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77d191e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f77cc0b5110 0x7f77cc06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77d191e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77cc0b5520 con 0x7f77cc15cae0 2024-04-25T11:21:11.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77d191e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f77cc0ce2a0 con 0x7f77cc0b5110 2024-04-25T11:21:11.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77d191e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f77cc0b4460 con 0x7f77cc160fc0 2024-04-25T11:21:11.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77ca7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc160fc0 0x7f77cc0b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77caffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f77cc15cae0 0x7f77cc0cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:39130/0 (socket says 172.21.15.106:39130) 2024-04-25T11:21:11.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77cb7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f77cc0b5110 0x7f77cc06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.682+0000 7f77caffd640 1 -- 172.21.15.106:0/1136405889 learned_addr learned my addr 172.21.15.106:0/1136405889 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:11.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77cb7fe640 1 -- 172.21.15.106:0/1136405889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc160fc0 msgr2=0x7f77cc0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:11.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77cb7fe640 1 --2- 172.21.15.106:0/1136405889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc160fc0 0x7f77cc0b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77cb7fe640 1 -- 172.21.15.106:0/1136405889 >> v1:172.21.15.134:6789/0 conn(0x7f77cc15cae0 legacy=0x7f77cc0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77cb7fe640 1 -- 172.21.15.106:0/1136405889 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc1613d0 con 0x7f77cc0b5110 2024-04-25T11:21:11.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77cb7fe640 1 --2- 172.21.15.106:0/1136405889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc0b5110 0x7f77cc06c820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77c40097c0 tx=0x7f77c401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b79909e86263d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77c9ffb640 1 -- 172.21.15.106:0/1136405889 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40b0070 con 0x7f77cc0b5110 2024-04-25T11:21:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77c9ffb640 1 -- 172.21.15.106:0/1136405889 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c40ab600 con 0x7f77cc0b5110 2024-04-25T11:21:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77c9ffb640 1 -- 172.21.15.106:0/1136405889 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40ab9d0 con 0x7f77cc0b5110 2024-04-25T11:21:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77d191e640 1 -- 172.21.15.106:0/1136405889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc0b5110 msgr2=0x7f77cc06c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77d191e640 1 --2- 172.21.15.106:0/1136405889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc0b5110 0x7f77cc06c820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77c40097c0 tx=0x7f77c401c410 comp rx=0 tx=0).stop 2024-04-25T11:21:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77d191e640 1 -- 172.21.15.106:0/1136405889 shutdown_connections 2024-04-25T11:21:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77d191e640 1 --2- 172.21.15.106:0/1136405889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc0b5110 0x7f77cc06c820 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77d191e640 1 --2- 172.21.15.106:0/1136405889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc160fc0 0x7f77cc0b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77d191e640 1 -- 172.21.15.106:0/1136405889 >> 172.21.15.106:0/1136405889 conn(0x7f77cc0b3560 msgr2=0x7f77cc0b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77d191e640 1 -- 172.21.15.106:0/1136405889 shutdown_connections 2024-04-25T11:21:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.686+0000 7f77d191e640 1 -- 172.21.15.106:0/1136405889 wait complete. 2024-04-25T11:21:11.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77d191e640 1 Processor -- start 2024-04-25T11:21:11.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77d191e640 1 -- start start 2024-04-25T11:21:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77d191e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc0b5110 0x7f77cc153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77d191e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc15cae0 0x7f77cc149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77d191e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77cc160fc0 0x7f77cc1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77d191e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc06d170 con 0x7f77cc160fc0 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77d191e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc0b4740 con 0x7f77cc15cae0 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77d191e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f77cc0cfde0 con 0x7f77cc0b5110 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.690+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77cc160fc0 0x7f77cc1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77cc160fc0 0x7f77cc1524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36318/0 (socket says 172.21.15.106:36318) 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77cb7fe640 1 -- 172.21.15.106:0/791447018 learned_addr learned my addr 172.21.15.106:0/791447018 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77ca7fc640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc15cae0 0x7f77cc149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77caffd640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc0b5110 0x7f77cc153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77cb7fe640 1 -- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc0b5110 msgr2=0x7f77cc153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77cb7fe640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc0b5110 0x7f77cc153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77cb7fe640 1 -- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc15cae0 msgr2=0x7f77cc149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77cb7fe640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc15cae0 0x7f77cc149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77cb7fe640 1 -- 172.21.15.106:0/791447018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc156eb0 con 0x7f77cc160fc0 2024-04-25T11:21:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77ca7fc640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc15cae0 0x7f77cc149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:11.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77cb7fe640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77cc160fc0 0x7f77cc1524d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f77c40ae040 tx=0x7f77c40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77c3fff640 1 -- 172.21.15.106:0/791447018 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40b0040 con 0x7f77cc160fc0 2024-04-25T11:21:11.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77c3fff640 1 -- 172.21.15.106:0/791447018 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c40b9070 con 0x7f77cc160fc0 2024-04-25T11:21:11.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77c3fff640 1 -- 172.21.15.106:0/791447018 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40b4650 con 0x7f77cc160fc0 2024-04-25T11:21:11.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77cc1f7a10 con 0x7f77cc160fc0 2024-04-25T11:21:11.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77cc156330 con 0x7f77cc160fc0 2024-04-25T11:21:11.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77cc1f7be0 con 0x7f77cc160fc0 2024-04-25T11:21:11.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77c3fff640 1 -- 172.21.15.106:0/791447018 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77c40b47f0 con 0x7f77cc160fc0 2024-04-25T11:21:11.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77c3fff640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f77ac06f430 0x7f77ac071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77c3fff640 1 -- 172.21.15.106:0/791447018 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f77c41289d0 con 0x7f77cc160fc0 2024-04-25T11:21:11.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77caffd640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f77ac06f430 0x7f77ac071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.694+0000 7f77caffd640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f77ac06f430 0x7f77ac071940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0012f0 tx=0x7f77bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.702+0000 7f77c3fff640 1 -- 172.21.15.106:0/791447018 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77cc156eb0 con 0x7f77cc160fc0 2024-04-25T11:21:11.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.850+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f77cc050750 con 0x7f77ac06f430 2024-04-25T11:21:11.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.850+0000 7f77c3fff640 1 -- 172.21.15.106:0/791447018 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f77cc050750 con 0x7f77ac06f430 2024-04-25T11:21:11.851 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T11:21:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f77ac06f430 msgr2=0x7f77ac071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f77ac06f430 0x7f77ac071940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0012f0 tx=0x7f77bc009040 comp rx=0 tx=0).stop 2024-04-25T11:21:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77cc160fc0 msgr2=0x7f77cc1524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77cc160fc0 0x7f77cc1524d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f77c40ae040 tx=0x7f77c40191b0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 shutdown_connections 2024-04-25T11:21:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f77ac06f430 0x7f77ac071940 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77cc160fc0 0x7f77cc1524d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77cc15cae0 0x7f77cc149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 --2- 172.21.15.106:0/791447018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77cc0b5110 0x7f77cc153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 >> 172.21.15.106:0/791447018 conn(0x7f77cc0b3560 msgr2=0x7f77cc1602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 shutdown_connections 2024-04-25T11:21:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:11.854+0000 7f77d191e640 1 -- 172.21.15.106:0/791447018 wait complete. 2024-04-25T11:21:11.870 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:21:12.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.002+0000 7f13deab0640 1 Processor -- start 2024-04-25T11:21:12.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.002+0000 7f13deab0640 1 -- start start 2024-04-25T11:21:12.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13deab0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d80cef60 0x7f13d8073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13deab0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d80ce050 con 0x7f13d80d0ce0 2024-04-25T11:21:12.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13deab0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d815e6b0 con 0x7f13d815e0a0 2024-04-25T11:21:12.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13deab0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13d815e990 con 0x7f13d80cef60 2024-04-25T11:21:12.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13dd04d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f13d80d0ce0 0x7f13d80d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45956/0 (socket says 172.21.15.106:45956) 2024-04-25T11:21:12.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13dd04d640 1 -- 172.21.15.106:0/2039325423 learned_addr learned my addr 172.21.15.106:0/2039325423 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:12.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d7fff640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d80cef60 0x7f13d8073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2339918738 0 0) 0x7f13d80ce050 con 0x7f13d80d0ce0 2024-04-25T11:21:12.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13c4003530 con 0x7f13d80d0ce0 2024-04-25T11:21:12.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3893213881 0 0) 0x7f13c4003530 con 0x7f13d80d0ce0 2024-04-25T11:21:12.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d80cef60 msgr2=0x7f13d8073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d80cef60 0x7f13d8073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 >> v1:172.21.15.134:6789/0 conn(0x7f13d815e0a0 legacy=0x7f13d80ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d815ec70 con 0x7f13d80d0ce0 2024-04-25T11:21:12.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4116766115 0 0) 0x7f13d0001680 con 0x7f13d80d0ce0 2024-04-25T11:21:12.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 >> v1:172.21.15.106:6789/0 conn(0x7f13d80d0ce0 legacy=0x7f13d80d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40039f0 0x7f13c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c4004380 0x7f13c40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c40050d0 0x7f13c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13d815e990 con 0x7f13c4004380 2024-04-25T11:21:12.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13c400bf70 con 0x7f13c40050d0 2024-04-25T11:21:12.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13c400c170 con 0x7f13c40039f0 2024-04-25T11:21:12.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d7fff640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c4004380 0x7f13c40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13dc84c640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40039f0 0x7f13c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13dd04d640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c40050d0 0x7f13c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d7fff640 1 -- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40039f0 msgr2=0x7f13c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d7fff640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40039f0 0x7f13c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d7fff640 1 -- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c40050d0 msgr2=0x7f13c400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d7fff640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c40050d0 0x7f13c400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.006+0000 7f13d7fff640 1 -- 172.21.15.106:0/2039325423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c4003530 con 0x7f13c4004380 2024-04-25T11:21:12.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13d7fff640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c4004380 0x7f13c40048d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009a90 tx=0x7f13c001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331cb988793f32af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00b2070 con 0x7f13c4004380 2024-04-25T11:21:12.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c00a5530 con 0x7f13c4004380 2024-04-25T11:21:12.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13d77fe640 1 -- 172.21.15.106:0/2039325423 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0009690 con 0x7f13c4004380 2024-04-25T11:21:12.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13deab0640 1 -- 172.21.15.106:0/2039325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c4004380 msgr2=0x7f13c40048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13deab0640 1 --2- 172.21.15.106:0/2039325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c4004380 0x7f13c40048d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009a90 tx=0x7f13c001ca50 comp rx=0 tx=0).stop 2024-04-25T11:21:12.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13dd04d640 1 -- 172.21.15.106:0/2039325423 reap_dead start 2024-04-25T11:21:12.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13deab0640 1 -- 172.21.15.106:0/2039325423 shutdown_connections 2024-04-25T11:21:12.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13deab0640 1 -- 172.21.15.106:0/2039325423 >> 172.21.15.106:0/2039325423 conn(0x7f13d806c7c0 msgr2=0x7f13d806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:12.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13deab0640 1 -- 172.21.15.106:0/2039325423 shutdown_connections 2024-04-25T11:21:12.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13deab0640 1 -- 172.21.15.106:0/2039325423 wait complete. 2024-04-25T11:21:12.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13deab0640 1 Processor -- start 2024-04-25T11:21:12.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.010+0000 7f13deab0640 1 -- start start 2024-04-25T11:21:12.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004380 0x7f13d8138db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13d814db60 0x7f13d814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d814df70 0x7f13d81500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13d80d0860 con 0x7f13d814db60 2024-04-25T11:21:12.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13d80d06e0 con 0x7f13c4004380 2024-04-25T11:21:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13d815f270 con 0x7f13d814df70 2024-04-25T11:21:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13d814db60 0x7f13d814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13d814db60 0x7f13d814fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36334/0 (socket says 172.21.15.106:36334) 2024-04-25T11:21:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 -- 172.21.15.106:0/1563288468 learned_addr learned my addr 172.21.15.106:0/1563288468 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13dc84c640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004380 0x7f13d8138db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13dd04d640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d814df70 0x7f13d81500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 -- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d814df70 msgr2=0x7f13d81500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d814df70 0x7f13d81500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 -- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004380 msgr2=0x7f13d8138db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004380 0x7f13d8138db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 -- 172.21.15.106:0/1563288468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d81392f0 con 0x7f13d814db60 2024-04-25T11:21:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13dc84c640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004380 0x7f13d8138db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d7fff640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13d814db60 0x7f13d814fbb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f13c001c880 tx=0x7f13c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d57fa640 1 -- 172.21.15.106:0/1563288468 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00b2040 con 0x7f13d814db60 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d57fa640 1 -- 172.21.15.106:0/1563288468 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c00b7660 con 0x7f13d814db60 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d814e770 con 0x7f13d814db60 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d57fa640 1 -- 172.21.15.106:0/1563288468 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00b7a10 con 0x7f13d814db60 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d814ed30 con 0x7f13d814db60 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d814e4c0 con 0x7f13d814db60 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.014+0000 7f13d57fa640 1 -- 172.21.15.106:0/1563288468 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13c00c1060 con 0x7f13d814db60 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.018+0000 7f13d57fa640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f13b806f400 0x7f13b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.018+0000 7f13d57fa640 1 -- 172.21.15.106:0/1563288468 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f13c0128fc0 con 0x7f13d814db60 2024-04-25T11:21:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.018+0000 7f13dc84c640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f13b806f400 0x7f13b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.018+0000 7f13dc84c640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f13b806f400 0x7f13b8071910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13c8001d90 tx=0x7f13c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.022+0000 7f13d57fa640 1 -- 172.21.15.106:0/1563288468 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13c0016760 con 0x7f13d814db60 2024-04-25T11:21:12.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.166+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13d81418f0 con 0x7f13b806f400 2024-04-25T11:21:12.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13d57fa640 1 -- 172.21.15.106:0/1563288468 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13447 (secure 0 0 0) 0x7f13d81418f0 con 0x7f13b806f400 2024-04-25T11:21:12.169 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:21:12.169 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f13b806f400 msgr2=0x7f13b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f13b806f400 0x7f13b8071910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13c8001d90 tx=0x7f13c8009040 comp rx=0 tx=0).stop 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13d814db60 msgr2=0x7f13d814fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13d814db60 0x7f13d814fbb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f13c001c880 tx=0x7f13c4005840 comp rx=0 tx=0).stop 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 shutdown_connections 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13d814df70 0x7f13d81500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f13b806f400 0x7f13b8071910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13d814db60 0x7f13d814fbb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 --2- 172.21.15.106:0/1563288468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004380 0x7f13d8138db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 >> 172.21.15.106:0/1563288468 conn(0x7f13d806c7c0 msgr2=0x7f13d8140e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:12.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 shutdown_connections 2024-04-25T11:21:12.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:12.170+0000 7f13deab0640 1 -- 172.21.15.106:0/1563288468 wait complete. 2024-04-25T11:21:12.189 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-04-25T11:21:12.125155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220596,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754124,"statfs":{"total":773094113280,"available":772868222976,"internally_reserved":0,"allocated":4530176,"data_stored":2494792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33554432,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":-9,"num_read_kb":-8192,"num_write":-8,"num_write_kb":-32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.625247"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":140,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T11:21:05.433078+0000","last_change":"2024-04-25T11:19:15.522583+0000","last_active":"2024-04-25T11:21:05.433078+0000","last_peered":"2024-04-25T11:21:05.433078+0000","last_clean":"2024-04-25T11:21:05.433078+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:21:05.433078+0000","last_undegraded":"2024-04-25T11:21:05.433078+0000","last_fullsized":"2024-04-25T11:21:05.433078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:14.302134+0000","last_clean_scrub_stamp":"2024-04-25T11:19:14.302134+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:05.863005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:21:12.194 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T11:19:14.302134+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=19, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=21, tm_sec=6, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:21:12.194 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T11:21:25.317 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:25.320 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:32.195 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:21:32.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e55bfe640 1 Processor -- start 2024-04-25T11:21:32.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e55bfe640 1 -- start start 2024-04-25T11:21:32.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e55bfe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e500d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e55bfe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e500b5a80 0x7f6e500b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e55bfe640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e501573a0 con 0x7f6e5006fea0 2024-04-25T11:21:32.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e55bfe640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6e500d1f90 con 0x7f6e500b5a80 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e55bfe640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6e500b4810 con 0x7f6e50156f90 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e500b5a80 0x7f6e500b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e500b5a80 0x7f6e500b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51420/0 (socket says 172.21.15.106:51420) 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4ffff640 1 -- 172.21.15.106:0/1783219322 learned_addr learned my addr 172.21.15.106:0/1783219322 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4f7fe640 1 --2- 172.21.15.106:0/1783219322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e500d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4ffff640 1 -- 172.21.15.106:0/1783219322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 msgr2=0x7f6e500d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4ffff640 1 --2- 172.21.15.106:0/1783219322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e500d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4ffff640 1 -- 172.21.15.106:0/1783219322 >> v1:172.21.15.134:6789/0 conn(0x7f6e5006fea0 legacy=0x7f6e500702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4ffff640 1 -- 172.21.15.106:0/1783219322 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5015d220 con 0x7f6e500b5a80 2024-04-25T11:21:32.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.354+0000 7f6e4f7fe640 1 --2- 172.21.15.106:0/1783219322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e500d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:32.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e4ffff640 1 --2- 172.21.15.106:0/1783219322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e500b5e90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800d150 tx=0x7f6e4801ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6125ba08e31bcbd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:32.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e4e7fc640 1 -- 172.21.15.106:0/1783219322 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800b2d0 con 0x7f6e500b5a80 2024-04-25T11:21:32.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e4e7fc640 1 -- 172.21.15.106:0/1783219322 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e4800b470 con 0x7f6e500b5a80 2024-04-25T11:21:32.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e4e7fc640 1 -- 172.21.15.106:0/1783219322 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e480a7a80 con 0x7f6e500b5a80 2024-04-25T11:21:32.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e55bfe640 1 -- 172.21.15.106:0/1783219322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 msgr2=0x7f6e500b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/1783219322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e500b5e90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800d150 tx=0x7f6e4801ebc0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e55bfe640 1 -- 172.21.15.106:0/1783219322 shutdown_connections 2024-04-25T11:21:32.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/1783219322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e500b5e90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/1783219322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e500d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e55bfe640 1 -- 172.21.15.106:0/1783219322 >> 172.21.15.106:0/1783219322 conn(0x7f6e5006c7c0 msgr2=0x7f6e5006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:32.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e55bfe640 1 -- 172.21.15.106:0/1783219322 shutdown_connections 2024-04-25T11:21:32.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.358+0000 7f6e55bfe640 1 -- 172.21.15.106:0/1783219322 wait complete. 2024-04-25T11:21:32.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 Processor -- start 2024-04-25T11:21:32.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 -- start start 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5006fea0 0x7f6e50142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e50162e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e501603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5015dda0 con 0x7f6e500b5a80 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5015ced0 con 0x7f6e5006fea0 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6e500d1290 con 0x7f6e50156f90 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e50162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e50162e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51436/0 (socket says 172.21.15.106:51436) 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 -- 172.21.15.106:0/3900704245 learned_addr learned my addr 172.21.15.106:0/3900704245 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4ffff640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e501603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4f7fe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5006fea0 0x7f6e50142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 -- 172.21.15.106:0/3900704245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 msgr2=0x7f6e501603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e501603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 -- 172.21.15.106:0/3900704245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5006fea0 msgr2=0x7f6e50142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5006fea0 0x7f6e50142c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 -- 172.21.15.106:0/3900704245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e50162a50 con 0x7f6e500b5a80 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e4effd640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e50162e60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400b410 tx=0x7f6e4400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e37fff640 1 -- 172.21.15.106:0/3900704245 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44015030 con 0x7f6e500b5a80 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e37fff640 1 -- 172.21.15.106:0/3900704245 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e44004020 con 0x7f6e500b5a80 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e501619b0 con 0x7f6e500b5a80 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e37fff640 1 -- 172.21.15.106:0/3900704245 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44004410 con 0x7f6e500b5a80 2024-04-25T11:21:32.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e50161f70 con 0x7f6e500b5a80 2024-04-25T11:21:32.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.362+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e500b1060 con 0x7f6e500b5a80 2024-04-25T11:21:32.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.366+0000 7f6e37fff640 1 -- 172.21.15.106:0/3900704245 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6e44018070 con 0x7f6e500b5a80 2024-04-25T11:21:32.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.366+0000 7f6e37fff640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6e3c06f400 0x7f6e3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.366+0000 7f6e37fff640 1 -- 172.21.15.106:0/3900704245 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f6e4408f2a0 con 0x7f6e500b5a80 2024-04-25T11:21:32.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.366+0000 7f6e4f7fe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6e3c06f400 0x7f6e3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.366+0000 7f6e4f7fe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6e3c06f400 0x7f6e3c071910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6e40006fd0 tx=0x7f6e40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:32.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.370+0000 7f6e37fff640 1 -- 172.21.15.106:0/3900704245 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6e4405c570 con 0x7f6e500b5a80 2024-04-25T11:21:32.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.522+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 --> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e500b27e0 con 0x7f6e3c06f400 2024-04-25T11:21:32.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e37fff640 1 -- 172.21.15.106:0/3900704245 <== mgr.4104 v2:172.21.15.106:6832/3760950393 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13341 (secure 0 0 0) 0x7f6e500b27e0 con 0x7f6e3c06f400 2024-04-25T11:21:32.524 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:21:32.524 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T11:21:32.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6e3c06f400 msgr2=0x7f6e3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6e3c06f400 0x7f6e3c071910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6e40006fd0 tx=0x7f6e40008040 comp rx=0 tx=0).stop 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 msgr2=0x7f6e50162e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e50162e60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400b410 tx=0x7f6e4400b8d0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 shutdown_connections 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e50156f90 0x7f6e501603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6e3c06f400 0x7f6e3c071910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e500b5a80 0x7f6e50162e60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 --2- 172.21.15.106:0/3900704245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5006fea0 0x7f6e50142c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 >> 172.21.15.106:0/3900704245 conn(0x7f6e5006c7c0 msgr2=0x7f6e5006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 shutdown_connections 2024-04-25T11:21:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.526+0000 7f6e55bfe640 1 -- 172.21.15.106:0/3900704245 wait complete. 2024-04-25T11:21:32.544 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-04-25T11:21:32.130991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":224748,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749972,"statfs":{"total":773094113280,"available":772863971328,"internally_reserved":0,"allocated":4530176,"data_stored":2494792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003619"},"pg_stats":[{"pgid":"1.0","version":"36'74","reported_seq":152,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T11:21:11.958544+0000","last_change":"2024-04-25T11:21:11.958544+0000","last_active":"2024-04-25T11:21:11.958544+0000","last_peered":"2024-04-25T11:21:11.958544+0000","last_clean":"2024-04-25T11:21:11.958544+0000","last_became_active":"2024-04-25T11:19:15.519821+0000","last_became_peered":"2024-04-25T11:19:15.519821+0000","last_unstale":"2024-04-25T11:21:11.958544+0000","last_undegraded":"2024-04-25T11:21:11.958544+0000","last_fullsized":"2024-04-25T11:21:11.958544+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'74","last_scrub_stamp":"2024-04-25T11:21:11.958246+0000","last_deep_scrub":"36'74","last_deep_scrub_stamp":"2024-04-25T11:21:11.958246+0000","last_clean_scrub_stamp":"2024-04-25T11:21:11.958246+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:27.080226+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:21:32.545 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T11:21:32.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7c998640 1 Processor -- start 2024-04-25T11:21:32.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7c998640 1 -- start start 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7c998640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdf78143fa0 0x7fdf780cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7c998640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdf780d1e50 0x7fdf780b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7c998640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf78073300 con 0x7fdf78072a20 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7c998640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdf780735e0 con 0x7fdf780d1e50 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7c998640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdf780738c0 con 0x7fdf78143fa0 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf76d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdf780d1e50 0x7fdf780b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf75d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdf78072a20 0x7fdf780d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47526/0 (socket says 172.21.15.106:47526) 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf75d9b640 1 -- 172.21.15.106:0/4115451166 learned_addr learned my addr 172.21.15.106:0/4115451166 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7659c640 1 --2- 172.21.15.106:0/4115451166 >> v2:172.21.15.134:3300/0 conn(0x7fdf78143fa0 0x7fdf780cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf76d9d640 1 -- 172.21.15.106:0/4115451166 >> v2:172.21.15.134:3300/0 conn(0x7fdf78143fa0 msgr2=0x7fdf780cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf76d9d640 1 --2- 172.21.15.106:0/4115451166 >> v2:172.21.15.134:3300/0 conn(0x7fdf78143fa0 0x7fdf780cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf76d9d640 1 -- 172.21.15.106:0/4115451166 >> v1:172.21.15.106:6789/0 conn(0x7fdf78072a20 legacy=0x7fdf780d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf7559a640 1 -- 172.21.15.106:0/4115451166 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1429453372 0 0) 0x7fdf78073300 con 0x7fdf78072a20 2024-04-25T11:21:32.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.698+0000 7fdf76d9d640 1 -- 172.21.15.106:0/4115451166 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf780b5040 con 0x7fdf780d1e50 2024-04-25T11:21:32.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf76d9d640 1 --2- 172.21.15.106:0/4115451166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf780d1e50 0x7fdf780b5840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdf78070f30 tx=0x7fdf7001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe06acacd8791b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:32.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7559a640 1 -- 172.21.15.106:0/4115451166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf700b2070 con 0x7fdf780d1e50 2024-04-25T11:21:32.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7559a640 1 -- 172.21.15.106:0/4115451166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf700a59e0 con 0x7fdf780d1e50 2024-04-25T11:21:32.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7559a640 1 -- 172.21.15.106:0/4115451166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf700a5db0 con 0x7fdf780d1e50 2024-04-25T11:21:32.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 -- 172.21.15.106:0/4115451166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf780d1e50 msgr2=0x7fdf780b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 --2- 172.21.15.106:0/4115451166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf780d1e50 0x7fdf780b5840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdf78070f30 tx=0x7fdf7001c900 comp rx=0 tx=0).stop 2024-04-25T11:21:32.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 -- 172.21.15.106:0/4115451166 shutdown_connections 2024-04-25T11:21:32.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 --2- 172.21.15.106:0/4115451166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf780d1e50 0x7fdf780b5840 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 --2- 172.21.15.106:0/4115451166 >> v2:172.21.15.134:3300/0 conn(0x7fdf78143fa0 0x7fdf780cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 -- 172.21.15.106:0/4115451166 >> 172.21.15.106:0/4115451166 conn(0x7fdf7806c9f0 msgr2=0x7fdf7806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:32.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 -- 172.21.15.106:0/4115451166 shutdown_connections 2024-04-25T11:21:32.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 -- 172.21.15.106:0/4115451166 wait complete. 2024-04-25T11:21:32.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 Processor -- start 2024-04-25T11:21:32.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.702+0000 7fdf7c998640 1 -- start start 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf78072a20 0x7fdf7814e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf78143fa0 0x7fdf7815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf781557b0 0x7fdf7814ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7816ad20 con 0x7fdf781557b0 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7816aba0 con 0x7fdf78072a20 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7816aea0 con 0x7fdf78143fa0 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf781557b0 0x7fdf7814ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf781557b0 0x7fdf7814ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51452/0 (socket says 172.21.15.106:51452) 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 -- 172.21.15.106:0/2756170060 learned_addr learned my addr 172.21.15.106:0/2756170060 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7659c640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf78072a20 0x7fdf7814e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf75d9b640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf78143fa0 0x7fdf7815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 -- 172.21.15.106:0/2756170060 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf78143fa0 msgr2=0x7fdf7815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf78143fa0 0x7fdf7815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 -- 172.21.15.106:0/2756170060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf78072a20 msgr2=0x7fdf7814e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf78072a20 0x7fdf7814e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 -- 172.21.15.106:0/2756170060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf78154710 con 0x7fdf781557b0 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf76d9d640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf781557b0 0x7fdf7814ff10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdf700b0040 tx=0x7fdf700196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf6effd640 1 -- 172.21.15.106:0/2756170060 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf700b2050 con 0x7fdf781557b0 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf6effd640 1 -- 172.21.15.106:0/2756170060 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf700be070 con 0x7fdf781557b0 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf781549f0 con 0x7fdf781557b0 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf6effd640 1 -- 172.21.15.106:0/2756170060 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf700adc00 con 0x7fdf781557b0 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf78154fb0 con 0x7fdf781557b0 2024-04-25T11:21:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf78154460 con 0x7fdf781557b0 2024-04-25T11:21:32.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.706+0000 7fdf6effd640 1 -- 172.21.15.106:0/2756170060 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdf700b9450 con 0x7fdf781557b0 2024-04-25T11:21:32.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.710+0000 7fdf6effd640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdf5806f430 0x7fdf58071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:32.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.710+0000 7fdf6effd640 1 -- 172.21.15.106:0/2756170060 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fdf70129dc0 con 0x7fdf781557b0 2024-04-25T11:21:32.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.710+0000 7fdf7659c640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdf5806f430 0x7fdf58071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:32.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.710+0000 7fdf7659c640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdf5806f430 0x7fdf58071940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800b930 tx=0x7fdf68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:32.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.714+0000 7fdf6effd640 1 -- 172.21.15.106:0/2756170060 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdf700c7020 con 0x7fdf781557b0 2024-04-25T11:21:32.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.870+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fdf78050750 con 0x7fdf781557b0 2024-04-25T11:21:32.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.878+0000 7fdf6effd640 1 -- 172.21.15.106:0/2756170060 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fdf70016520 con 0x7fdf781557b0 2024-04-25T11:21:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.878+0000 7fdf6effd640 1 -- 172.21.15.106:0/2756170060 <== mon.0 v2:172.21.15.106:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fdf700b3040 con 0x7fdf781557b0 2024-04-25T11:21:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdf5806f430 msgr2=0x7fdf58071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdf5806f430 0x7fdf58071940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800b930 tx=0x7fdf68008040 comp rx=0 tx=0).stop 2024-04-25T11:21:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf781557b0 msgr2=0x7fdf7814ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf781557b0 0x7fdf7814ff10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdf700b0040 tx=0x7fdf700196a0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 shutdown_connections 2024-04-25T11:21:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7fdf5806f430 0x7fdf58071940 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf781557b0 0x7fdf7814ff10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf78143fa0 0x7fdf7815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 --2- 172.21.15.106:0/2756170060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf78072a20 0x7fdf7814e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 >> 172.21.15.106:0/2756170060 conn(0x7fdf7806c9f0 msgr2=0x7fdf78144800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:32.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 shutdown_connections 2024-04-25T11:21:32.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:32.882+0000 7fdf7c998640 1 -- 172.21.15.106:0/2756170060 wait complete. 2024-04-25T11:21:32.900 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:21:32.900 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:33.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.042+0000 7f6bb40b2640 1 Processor -- start 2024-04-25T11:21:33.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb40b2640 1 -- start start 2024-04-25T11:21:33.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb40b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac156f90 0x7f6bac0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb40b2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bac06fea0 0x7f6bac0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb40b2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6bac0b5a80 0x7f6bac0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb40b2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6bac0d1f90 con 0x7f6bac06fea0 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb40b2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6bac0b4810 con 0x7f6bac0b5a80 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb40b2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6bac0707f0 con 0x7f6bac156f90 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb164d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bac06fea0 0x7f6bac0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb164d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bac06fea0 0x7f6bac0702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51462/0 (socket says 172.21.15.106:51462) 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb164d640 1 -- 172.21.15.106:0/3708806126 learned_addr learned my addr 172.21.15.106:0/3708806126 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb1e4e640 1 --2- 172.21.15.106:0/3708806126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac156f90 0x7f6bac0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb264f640 1 --2- 172.21.15.106:0/3708806126 >> v2:172.21.15.134:3300/0 conn(0x7f6bac0b5a80 0x7f6bac0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb164d640 1 -- 172.21.15.106:0/3708806126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac156f90 msgr2=0x7f6bac0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb164d640 1 --2- 172.21.15.106:0/3708806126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac156f90 0x7f6bac0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb164d640 1 -- 172.21.15.106:0/3708806126 >> v2:172.21.15.134:3300/0 conn(0x7f6bac0b5a80 msgr2=0x7f6bac0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:33.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb164d640 1 --2- 172.21.15.106:0/3708806126 >> v2:172.21.15.134:3300/0 conn(0x7f6bac0b5a80 0x7f6bac0b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.046+0000 7f6bb164d640 1 -- 172.21.15.106:0/3708806126 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac1573a0 con 0x7f6bac06fea0 2024-04-25T11:21:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb164d640 1 --2- 172.21.15.106:0/3708806126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac06fea0 0x7f6bac0702b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6bac070a60 tx=0x7f6b9c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43272d8675a047dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb0e4c640 1 -- 172.21.15.106:0/3708806126 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0b1070 con 0x7f6bac06fea0 2024-04-25T11:21:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb0e4c640 1 -- 172.21.15.106:0/3708806126 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6b9c0ac620 con 0x7f6bac06fea0 2024-04-25T11:21:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb0e4c640 1 -- 172.21.15.106:0/3708806126 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0aca10 con 0x7f6bac06fea0 2024-04-25T11:21:33.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3708806126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac06fea0 msgr2=0x7f6bac0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3708806126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac06fea0 0x7f6bac0702b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6bac070a60 tx=0x7f6b9c01c4f0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3708806126 shutdown_connections 2024-04-25T11:21:33.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3708806126 >> v2:172.21.15.134:3300/0 conn(0x7f6bac0b5a80 0x7f6bac0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3708806126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac06fea0 0x7f6bac0702b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3708806126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac156f90 0x7f6bac0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3708806126 >> 172.21.15.106:0/3708806126 conn(0x7f6bac06c7c0 msgr2=0x7f6bac06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:33.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3708806126 shutdown_connections 2024-04-25T11:21:33.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.050+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3708806126 wait complete. 2024-04-25T11:21:33.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 Processor -- start 2024-04-25T11:21:33.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 -- start start 2024-04-25T11:21:33.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac06fea0 0x7f6bac159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac0b5a80 0x7f6bac14ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bac156f90 0x7f6bac072a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac15ced0 con 0x7f6bac0b5a80 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac15dac0 con 0x7f6bac156f90 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6bac15dda0 con 0x7f6bac06fea0 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac0b5a80 0x7f6bac14ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac0b5a80 0x7f6bac14ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51466/0 (socket says 172.21.15.106:51466) 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 -- 172.21.15.106:0/3282057318 learned_addr learned my addr 172.21.15.106:0/3282057318 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb264f640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bac156f90 0x7f6bac072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb1e4e640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac06fea0 0x7f6bac159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 -- 172.21.15.106:0/3282057318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac06fea0 msgr2=0x7f6bac159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac06fea0 0x7f6bac159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 -- 172.21.15.106:0/3282057318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bac156f90 msgr2=0x7f6bac072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bac156f90 0x7f6bac072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 -- 172.21.15.106:0/3282057318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac157d50 con 0x7f6bac0b5a80 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb164d640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac0b5a80 0x7f6bac14ecd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c01aba0 tx=0x7f6b9c019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6ba27fc640 1 -- 172.21.15.106:0/3282057318 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0b1040 con 0x7f6bac0b5a80 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6ba27fc640 1 -- 172.21.15.106:0/3282057318 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6b9c0ba070 con 0x7f6bac0b5a80 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac158030 con 0x7f6bac0b5a80 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6ba27fc640 1 -- 172.21.15.106:0/3282057318 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0b5880 con 0x7f6bac0b5a80 2024-04-25T11:21:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac1f7790 con 0x7f6bac0b5a80 2024-04-25T11:21:33.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.054+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac0cdf60 con 0x7f6bac0b5a80 2024-04-25T11:21:33.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.058+0000 7f6ba27fc640 1 -- 172.21.15.106:0/3282057318 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6b9c0ac4e0 con 0x7f6bac0b5a80 2024-04-25T11:21:33.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.058+0000 7f6ba27fc640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6b9006f400 0x7f6b90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.058+0000 7f6ba27fc640 1 -- 172.21.15.106:0/3282057318 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f6b9c129010 con 0x7f6bac0b5a80 2024-04-25T11:21:33.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.058+0000 7f6bb1e4e640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6b9006f400 0x7f6b90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.058+0000 7f6bb1e4e640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6b9006f400 0x7f6b90071910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d8d0 tx=0x7f6ba8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.062+0000 7f6ba27fc640 1 -- 172.21.15.106:0/3282057318 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6b9c016110 con 0x7f6bac0b5a80 2024-04-25T11:21:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.242+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bac166e90 con 0x7f6bac0b5a80 2024-04-25T11:21:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.242+0000 7f6ba27fc640 1 -- 172.21.15.106:0/3282057318 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3881 (secure 0 0 0) 0x7f6b9c0f6160 con 0x7f6bac0b5a80 2024-04-25T11:21:33.243 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T11:21:33.243 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3211345151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3211345151},{"type":"v1","addr":"172.21.15.134:6833","nonce":3211345151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1962020849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1962020849},{"type":"v1","addr":"172.21.15.106:6835","nonce":1962020849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3211705687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3211705687},{"type":"v1","addr":"172.21.15.134:6835","nonce":3211705687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3124863947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3124863947},{"type":"v1","addr":"172.21.15.106:6837","nonce":3124863947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-04-25T11:21:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 19 2024-04-25T11:21:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6b9006f400 msgr2=0x7f6b90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6b9006f400 0x7f6b90071910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d8d0 tx=0x7f6ba8008040 comp rx=0 tx=0).stop 2024-04-25T11:21:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac0b5a80 msgr2=0x7f6bac14ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac0b5a80 0x7f6bac14ecd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c01aba0 tx=0x7f6b9c019290 comp rx=0 tx=0).stop 2024-04-25T11:21:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 shutdown_connections 2024-04-25T11:21:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bac156f90 0x7f6bac072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:6832/3760950393,v1:172.21.15.106:6833/3760950393] conn(0x7f6b9006f400 0x7f6b90071910 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bac0b5a80 0x7f6bac14ecd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 --2- 172.21.15.106:0/3282057318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bac06fea0 0x7f6bac159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 >> 172.21.15.106:0/3282057318 conn(0x7f6bac06c7c0 msgr2=0x7f6bac1560d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 shutdown_connections 2024-04-25T11:21:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T11:21:33.246+0000 7f6bb40b2640 1 -- 172.21.15.106:0/3282057318 wait complete. 2024-04-25T11:21:33.263 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T11:21:33.263 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T11:21:33.264 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:21:33.264 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.286 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:21:33.286 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:21:33.286 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.348 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:21:33.348 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:21:33.348 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.363 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:21:33.364 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:21:33.364 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.421 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:21:33.421 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T11:21:33.421 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T11:21:33.421 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.493 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T11:21:33.493 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T11:21:33.493 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.590 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T11:21:33.591 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T11:21:33.591 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.655 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T11:21:33.655 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T11:21:33.656 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.755 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T11:21:33.755 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T11:21:33.755 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.839 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T11:21:33.839 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T11:21:33.839 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.925 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T11:21:33.925 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T11:21:33.925 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:33.994 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T11:21:33.994 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T11:21:33.994 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:34.081 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T11:21:34.081 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T11:21:34.082 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T11:21:34.082 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:34.112 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T11:21:34.112 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T11:21:34.112 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:34.142 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T11:21:34.142 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T11:21:34.142 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T11:21:34.142 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:34.172 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T11:21:34.173 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T11:21:34.173 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:34.208 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T11:21:34.209 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T11:21:34.209 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:34.283 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T11:21:34.283 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T11:21:34.283 DEBUG:teuthology.orchestra.run.smithi106:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T11:21:34.301 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:21:34.301 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T11:21:34.715 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:21:34.715 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T11:21:34.959 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:21:34.959 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T11:21:35.118 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:21:35.118 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T11:21:35.327 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:21:35.327 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T11:21:35.666 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:21:35.666 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T11:21:35.897 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:21:35.898 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T11:21:36.090 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:21:36.090 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T11:21:36.290 INFO:tasks.ceph:Archiving mon data... 2024-04-25T11:21:36.291 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/data/mon.a.tgz 2024-04-25T11:21:36.292 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-04-25T11:21:36.298 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.QmbluzQbwZ 2024-04-25T11:21:36.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QmbluzQbwZ 2024-04-25T11:21:36.414 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.QmbluzQbwZ 2024-04-25T11:21:36.485 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.QmbluzQbwZ is 285KB 2024-04-25T11:21:36.546 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.QmbluzQbwZ 2024-04-25T11:21:36.551 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/data/mon.b.tgz 2024-04-25T11:21:36.551 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-04-25T11:21:36.557 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.pJL46o2rxY 2024-04-25T11:21:36.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.pJL46o2rxY 2024-04-25T11:21:36.690 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.pJL46o2rxY 2024-04-25T11:21:36.755 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.pJL46o2rxY is 382KB 2024-04-25T11:21:36.817 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.pJL46o2rxY 2024-04-25T11:21:36.822 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/data/mon.c.tgz 2024-04-25T11:21:36.822 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-04-25T11:21:36.871 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.MGUgupvWUS 2024-04-25T11:21:36.871 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.MGUgupvWUS 2024-04-25T11:21:37.000 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.MGUgupvWUS 2024-04-25T11:21:37.063 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.MGUgupvWUS is 382KB 2024-04-25T11:21:37.125 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.MGUgupvWUS 2024-04-25T11:21:37.131 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T11:21:37.131 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:21:37.133 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:21:37.192 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:21:37.235 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:21:37.236 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T11:21:37.237 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/remote/smithi106/crash 2024-04-25T11:21:37.238 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:21:37.252 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/remote/smithi134/crash 2024-04-25T11:21:37.253 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:21:37.291 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T11:21:37.291 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:37.297 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:37.378 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:37.391 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:37.434 INFO:tasks.ceph:Compressing logs... 2024-04-25T11:21:37.434 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:21:37.437 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:21:37.456 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35988.log 2024-04-25T11:21:37.456 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36228.log 2024-04-25T11:21:37.457 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34128.log 2024-04-25T11:21:37.458 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35988.log: /var/log/ceph/ceph-client.admin.36228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39768.log 2024-04-25T11:21:37.458 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35988.log.gz 2024-04-25T11:21:37.458 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34128.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36228.log.gz 2024-04-25T11:21:37.459 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38950.log 2024-04-25T11:21:37.459 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34128.log.gz 2024-04-25T11:21:37.459 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39533.log 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38339.log 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39768.log.gz 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38950.log.gz 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38658.log 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39533.log.gz 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39299.log 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38339.log.gz 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38286.log 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38658.log.gz 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33507.log 2024-04-25T11:21:37.462 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38286.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39299.log.gz 2024-04-25T11:21:37.463 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33929.log 2024-04-25T11:21:37.463 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38286.log.gz 2024-04-25T11:21:37.463 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38391.log 2024-04-25T11:21:37.463 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33507.log.gz 2024-04-25T11:21:37.463 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36699.log 2024-04-25T11:21:37.463 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38391.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33929.log.gz 2024-04-25T11:21:37.463 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33311.log 2024-04-25T11:21:37.468 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38391.log.gz 2024-04-25T11:21:37.468 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34133.log 2024-04-25T11:21:37.468 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36699.log.gz 2024-04-25T11:21:37.468 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.33311.log: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T11:21:37.468 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-04-25T11:21:37.468 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33311.log.gz 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34133.log.gz 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37966.log 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35643.log 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34631.log 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39125.log 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37966.log.gz 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34631.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35643.log.gz 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34346.log 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34631.log.gz 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33402.log 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39663.log 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39125.log.gz 2024-04-25T11:21:37.469 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34346.log.gz 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33402.log.gz 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39273.log 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35962.log 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39663.log.gz 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27566.log 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39273.log.gz 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39429.log 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.27566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36568.log 2024-04-25T11:21:37.470 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35962.log.gz 2024-04-25T11:21:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27566.log.gz/var/log/ceph/ceph-client.admin.39429.log: 2024-04-25T11:21:37.475 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T11:21:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39429.log.gz 2024-04-25T11:21:37.475 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37121.log 2024-04-25T11:21:37.475 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33923.log 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36568.log.gz 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36595.log 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37121.log.gz 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33278.log 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33923.log.gz 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38474.log 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36595.log: /var/log/ceph/ceph-client.admin.33278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37303.log 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36595.log.gz 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33278.log.gz 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37888.log 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:gzip 88.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.38474.log.gz -- 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.35152.log 2024-04-25T11:21:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37303.log: /var/log/ceph/ceph-client.admin.37888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34138.log 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37303.log.gz 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35152.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37888.log.gz 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38630.log 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34138.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35152.log.gz 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34579.log 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34138.log.gz 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35297.log 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38630.log: gzip/var/log/ceph/ceph-client.admin.34579.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38794.log 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38630.log.gz 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35747.log 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34579.log.gz 2024-04-25T11:21:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35297.log.gz 2024-04-25T11:21:37.478 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36093.log 2024-04-25T11:21:37.478 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35747.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.38794.log.gz 2024-04-25T11:21:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36332.log 2024-04-25T11:21:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35747.log.gz 2024-04-25T11:21:37.484 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34683.log 2024-04-25T11:21:37.484 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33877.log 2024-04-25T11:21:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36093.log.gz 2024-04-25T11:21:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36332.log.gz 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34683.log: /var/log/ceph/ceph-client.admin.37992.log 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33877.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34683.log.gz 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35591.log 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33877.log.gz 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37832.log 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35591.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37992.log.gz 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35825.log 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37832.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35591.log.gz 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37862.log 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37832.log.gz 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33916.log 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-04-25T11:21:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37862.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35825.log.gz 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34319.log 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37862.log.gz 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33916.log.gz 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36490.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.34319.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.2.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34319.log.gz 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35669.log 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39689.log 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39351.log: /var/log/ceph/ceph-client.admin.35669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36410.log 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38044.log 2024-04-25T11:21:37.486 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35669.log.gz 2024-04-25T11:21:37.487 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39689.log.gz 2024-04-25T11:21:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34919.log 2024-04-25T11:21:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38044.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36410.log.gz 2024-04-25T11:21:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22758.log 2024-04-25T11:21:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5/var/log/ceph/ceph-client.admin.34919.log: --verbose -- /var/log/ceph/ceph-client.admin.38417.log 2024-04-25T11:21:37.487 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38044.log.gz 2024-04-25T11:21:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.tmp-client.admin.22758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38710.log 2024-04-25T11:21:37.487 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22758.log.gz 2024-04-25T11:21:37.488 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39099.log 2024-04-25T11:21:37.488 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.34919.log.gz 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38417.log.gz 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38710.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39099.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38710.log.gz 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33481.log 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39099.log.gz 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-mon.a.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38365.log 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33376.log 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34136.log 2024-04-25T11:21:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38365.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33481.log.gz 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39377.log 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38365.log.gz 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33376.log.gz 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37427.log 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34136.log.gz 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39585.log 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36938.log 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37427.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39377.log.gz 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35695.log 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37427.log.gz 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39585.log.gz 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36938.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36938.log.gz 2024-04-25T11:21:37.495 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37251.log 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36067.log 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37251.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35695.log.gz 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37147.log 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37251.log.gz 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39073.log 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36067.log.gz 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35355.log 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35799.log 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37147.log.gz 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39073.log.gz 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34974.log 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34135.log 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35355.log.gz 2024-04-25T11:21:37.496 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35799.log.gz 2024-04-25T11:21:37.497 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34553.log 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27731.log 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.27731.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27731.log.gz 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33523.log 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27851.log 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33642.log 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.1.33523.log: /var/log/ceph/ceph-client.admin.27851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27827.log 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27851.log.gz 2024-04-25T11:21:37.514 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.3.33642.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.27827.log: 90.4% -- replaced with /var/log/ceph/ceph-client.1.33523.log.gz 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27779.log 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27827.log.gz 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27707.log 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.5.log: 90.7% -- replaced with /var/log/ceph/ceph-client.3.33642.log.gz 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.27779.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27779.log.gz 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.27707.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27707.log.gz 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27755.log 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27882.log 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.27755.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T11:21:37.515 INFO:teuthology.orchestra.run.smithi134.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:21:37.516 INFO:teuthology.orchestra.run.smithi134.stderr: 88.8%/var/log/ceph/ceph-client.admin.27882.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:21:37.516 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27755.log.gz 2024-04-25T11:21:37.516 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T11:21:37.516 INFO:teuthology.orchestra.run.smithi134.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.27882.log.gz 2024-04-25T11:21:37.516 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.audit.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:21:37.516 INFO:teuthology.orchestra.run.smithi134.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T11:21:37.516 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T11:21:37.521 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T11:21:37.521 INFO:teuthology.orchestra.run.smithi134.stderr:gzip/var/log/ceph/ceph-mds.b.log: -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T11:21:37.521 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27803.log 2024-04-25T11:21:37.521 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.4.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T11:21:37.521 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.27803.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27803.log.gz 2024-04-25T11:21:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38742.log 2024-04-25T11:21:37.527 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34974.log.gz 2024-04-25T11:21:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33695.log 2024-04-25T11:21:37.527 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34135.log.gz 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39507.log 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34553.log.gz 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38742.log.gz 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37479.log 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33695.log: /var/log/ceph/ceph-client.admin.39507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38018.log 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33695.log.gz 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39507.log.gz 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38846.log 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37479.log.gz 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35381.log 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38018.log.gz 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35409.log 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37095.log 2024-04-25T11:21:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35381.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38846.log.gz 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35409.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35381.log.gz 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36306.log 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35409.log.gz 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39611.log 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37095.log: /var/log/ceph/ceph-client.admin.36306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35323.log 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6%/var/log/ceph/ceph-client.admin.39611.log: -- replaced with /var/log/ceph/ceph-client.admin.36306.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37095.log.gz 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35323.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39611.log.gz 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33563.log 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35323.log.gz 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34119.log 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36990.log 2024-04-25T11:21:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39325.log 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33563.log.gz/var/log/ceph/ceph-client.admin.34119.log: 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36990.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34119.log.gz 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35773.log 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36040.log 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36990.log.gz/var/log/ceph/ceph-client.admin.39325.log: 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36756.log 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39325.log.gz 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35773.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34347.log 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36040.log: 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36280.log 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36040.log.gz 2024-04-25T11:21:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34761.log 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34347.log: /var/log/ceph/ceph-client.admin.36280.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36782.log 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36756.log.gz 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36280.log.gz 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34347.log.gz 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35851.log 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34761.log.gz 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35851.log: gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-osd.3.log 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36782.log.gz 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35851.log.gz 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39744.log 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38150.log 2024-04-25T11:21:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.3.log: -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34841.log 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39637.log 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39744.log.gz 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.38150.log: /var/log/ceph/ceph-client.admin.34841.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34841.log.gz 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34709.log 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36436.log 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34709.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39637.log.gz 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34709.log.gz 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35721.log 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34787.log 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36436.log.gz/var/log/ceph/ceph-client.admin.35721.log: 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35721.log.gz 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38260.log 2024-04-25T11:21:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36730.log 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34787.log: /var/log/ceph/ceph-client.admin.38260.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34787.log.gz 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38260.log.gz 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36673.log 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36730.log: /var/log/ceph/ceph-osd.1.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36730.log.gz 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38976.log 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36673.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36673.log.gz 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33210.log 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38976.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38976.log.gz 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35936.log 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33210.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33210.log.gz 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33617.log 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T11:21:37.533 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37637.log 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33617.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35936.log.gz 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37069.log 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33617.log.gz 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38684.log 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37069.log: 95.6% -- replaced with /var/log/ceph/ceph-client.0.38150.log.gz 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37637.log.gz 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37069.log.gz 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36621.log 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27786.log 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36621.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38684.log.gz 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38443.log 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34336.log 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36621.log.gz/var/log/ceph/ceph-client.admin.27786.log: 2024-04-25T11:21:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38443.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27786.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36886.log 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.38443.log.gz 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34815.log 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34336.log.gz 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38820.log 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36886.log.gz 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33930.log 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33721.log 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34815.log.gz 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38820.log: /var/log/ceph/ceph-client.admin.33930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33931.log 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38820.log.gz 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33930.log.gz 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33721.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33721.log.gz 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39403.log 2024-04-25T11:21:37.535 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37225.log 2024-04-25T11:21:37.536 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39403.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33931.log.gz 2024-04-25T11:21:37.536 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33851.log 2024-04-25T11:21:37.543 INFO:teuthology.orchestra.run.smithi134.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T11:21:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36358.log 2024-04-25T11:21:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37225.log: /var/log/ceph/ceph-client.admin.33851.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39403.log.gz 2024-04-25T11:21:37.551 INFO:teuthology.orchestra.run.smithi106.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37225.log.gz 2024-04-25T11:21:37.551 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33851.log.gz 2024-04-25T11:21:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33429.log 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37663.log 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33429.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36358.log.gz 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35903.log 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33429.log.gz 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37663.log: /var/log/ceph/ceph.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37663.log.gz 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr: 84.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37017.log 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34131.log 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35903.log: /var/log/ceph/ceph-client.admin.37017.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35903.log.gz /var/log/ceph/ceph-client.admin.34340.log 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-25T11:21:37.552 INFO:teuthology.orchestra.run.smithi106.stderr:gzip 88.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.37533.log 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37017.log.gz 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34131.log: /var/log/ceph/ceph-client.admin.34340.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34131.log.gz 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34340.log.gz 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35617.log 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37780.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34657.log 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37533.log.gz 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35617.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35617.log.gz 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33773.log 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37611.log 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9%/var/log/ceph/ceph-client.admin.33773.log: -- replaced with /var/log/ceph/ceph-client.admin.34657.log.gz 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36808.log 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33773.log.gz 2024-04-25T11:21:37.553 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37611.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37611.log.gz 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36014.log 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35487.log 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36808.log.gz/var/log/ceph/ceph-client.admin.36014.log: 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36119.log 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36014.log.gz 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35487.log: /var/log/ceph/ceph-client.admin.36119.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35487.log.gz 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36119.log.gz 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37914.log 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33643.log 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37914.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33825.log.gz 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36647.log 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37199.log 2024-04-25T11:21:37.554 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37914.log.gz/var/log/ceph/ceph-client.admin.33643.log: 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36647.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33643.log.gz 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36647.log.gz 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38924.log 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36860.log 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6%/var/log/ceph/ceph-client.admin.38924.log: -- replaced with /var/log/ceph/ceph-client.admin.37199.log.gz 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37940.log 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37559.log 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38924.log.gz 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36860.log: /var/log/ceph/ceph-client.admin.37940.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36860.log.gz 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37940.log.gz 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35877.log 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38604.log 2024-04-25T11:21:37.555 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9%/var/log/ceph/ceph-client.admin.35877.log: -- replaced with /var/log/ceph/ceph-client.admin.37559.log.gz 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39455.log 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35877.log.gz 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38604.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38604.log.gz 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36964.log 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36912.log 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39455.log: /var/log/ceph/ceph-client.admin.36964.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39455.log.gz -5 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33747.log 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36964.log.gz/var/log/ceph/ceph-client.admin.36912.log: 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36912.log.gz 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37361.log 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35461.log 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37361.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33747.log.gz 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39559.log 2024-04-25T11:21:37.556 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37361.log.gz 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35461.log: /var/log/ceph/ceph-client.admin.39559.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35461.log.gz 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39559.log.gz 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35513.log 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33533.log 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35513.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35513.log.gz 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34342.log 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33533.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4%/var/log/ceph/ceph-client.admin.34342.log: -- replaced with /var/log/ceph/ceph-client.admin.33533.log.gz 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34342.log.gz 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-04-25T11:21:37.557 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39151.log 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37173.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33799.log 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39151.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39151.log.gz 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34867.log 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33928.log 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34867.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33799.log.gz 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34867.log.gz 2024-04-25T11:21:37.566 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36171.log 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36202.log 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33928.log.gz/var/log/ceph/ceph-client.admin.36171.log: 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36171.log.gz 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38898.log 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38208.log 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38898.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36202.log.gz 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38898.log.gz 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36384.log 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33669.log 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36384.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38208.log.gz 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36384.log.gz 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39002.log 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33669.log.gz 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39002.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39002.log.gz 2024-04-25T11:21:37.567 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34735.log 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33455.log 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9%/var/log/ceph/ceph-client.admin.34735.log: -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34735.log.gz 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37277.log 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37507.log 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37277.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33455.log.gz 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37277.log.gz 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33919.log 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35247.log/var/log/ceph/ceph-client.admin.37507.log: 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3%/var/log/ceph/ceph-client.admin.33919.log: -- replaced with /var/log/ceph/ceph-client.admin.37507.log.gz 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33919.log.gz 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35539.log 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.2.35247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35102.log 2024-04-25T11:21:37.568 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35539.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35539.log.gz 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39195.log 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.2.35247.log.gz 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.35102.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.35102.log.gz 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37745.log 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38234.log 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2%/var/log/ceph/ceph-client.admin.37745.log: -- replaced with /var/log/ceph/ceph-client.admin.39195.log.gz 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37745.log.gz 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36254.log 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39715.log 2024-04-25T11:21:37.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36254.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38234.log.gz 2024-04-25T11:21:37.573 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36254.log.gz 2024-04-25T11:21:37.573 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34338.log 2024-04-25T11:21:37.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38312.log 2024-04-25T11:21:37.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34338.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39715.log.gz 2024-04-25T11:21:37.573 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34338.log.gz 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39247.log 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36834.log 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8%/var/log/ceph/ceph-client.admin.39247.log: -- replaced with /var/log/ceph/ceph-client.admin.38312.log.gz 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39247.log.gz 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37043.log 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.36834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27682.log 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36834.log.gz/var/log/ceph/ceph-client.admin.37043.log: 2024-04-25T11:21:37.575 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39028.log 2024-04-25T11:21:37.576 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37585.log 88.9% 2024-04-25T11:21:37.576 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37043.log.gz 2024-04-25T11:21:37.577 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39028.log: /var/log/ceph/ceph-client.admin.27682.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27682.log.gz 2024-04-25T11:21:37.577 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39028.log.gz 2024-04-25T11:21:37.577 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33591.log 2024-04-25T11:21:37.577 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37329.log 2024-04-25T11:21:37.578 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3%/var/log/ceph/ceph-client.admin.33591.log: -- replaced with /var/log/ceph/ceph-client.admin.37585.log.gzgzip 2024-04-25T11:21:37.578 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37689.log 2024-04-25T11:21:37.578 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3%/var/log/ceph/ceph-client.admin.37329.log: -- replaced with /var/log/ceph/ceph-client.admin.33591.log.gz 2024-04-25T11:21:37.578 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37329.log.gz 2024-04-25T11:21:37.578 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34527.log 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33927.log 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4%/var/log/ceph/ceph-client.admin.34527.log: -- replaced with /var/log/ceph/ceph-client.admin.37689.log.gz 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38872.log 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34527.log.gz 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33927.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33927.log.gz 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39794.log 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37717.log 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39794.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38872.log.gz 2024-04-25T11:21:37.580 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-04-25T11:21:37.583 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38094.log 2024-04-25T11:21:37.583 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39794.log.gz 2024-04-25T11:21:37.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37717.log: /var/log/ceph/ceph-client.admin.36542.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37717.log.gz 2024-04-25T11:21:37.583 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz 2024-04-25T11:21:37.583 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37453.log 2024-04-25T11:21:37.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36145.log 2024-04-25T11:21:37.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37453.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38094.log.gz 2024-04-25T11:21:37.583 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38530.log 2024-04-25T11:21:37.584 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34345.log 2024-04-25T11:21:37.584 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37453.log.gz/var/log/ceph/ceph-client.admin.36145.log: 2024-04-25T11:21:37.584 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.38530.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36145.log.gz 2024-04-25T11:21:37.584 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34893.log 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35435.log 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34893.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34345.log.gz 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34893.log.gz 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.0.38530.log.gz 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38768.log 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39481.log 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35435.log: /var/log/ceph/ceph-client.admin.38768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35565.log 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35435.log.gz 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39481.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38768.log.gz 2024-04-25T11:21:37.586 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39481.log.gz 2024-04-25T11:21:37.587 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37806.log 2024-04-25T11:21:37.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.35565.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35565.log.gz 2024-04-25T11:21:37.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37806.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37806.log.gz 2024-04-25T11:21:37.591 INFO:teuthology.orchestra.run.smithi106.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T11:21:37.594 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T11:21:37.610 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T11:21:37.611 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T11:21:37.637 INFO:teuthology.orchestra.run.smithi134.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T11:21:37.638 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T11:21:37.651 INFO:teuthology.orchestra.run.smithi106.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T11:21:37.654 INFO:teuthology.orchestra.run.smithi134.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T11:21:37.656 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T11:21:37.674 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T11:21:37.675 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-25T11:21:37.676 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m0.194s 2024-04-25T11:21:37.676 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m0.796s 2024-04-25T11:21:37.676 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.053s 2024-04-25T11:21:37.701 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T11:21:37.703 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-25T11:21:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m0.264s 2024-04-25T11:21:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m1.005s 2024-04-25T11:21:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m0.147s 2024-04-25T11:21:37.703 INFO:tasks.ceph:Archiving logs... 2024-04-25T11:21:37.703 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/remote/smithi106/log 2024-04-25T11:21:37.704 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:21:38.056 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/remote/smithi134/log 2024-04-25T11:21:38.057 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:21:38.167 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:21:38.195 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:21:38.196 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:21:38.199 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:21:38.261 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:21:38.297 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:21:38.297 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:21:38.300 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:21:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T11:21:38.311 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================== 2024-04-25T11:21:38.311 INFO:teuthology.orchestra.run.smithi106.stdout:+hv01.front.sepi 67.205.162.81 3 u 43 64 77 0.075 -2.111 0.596 2024-04-25T11:21:38.311 INFO:teuthology.orchestra.run.smithi106.stdout:*hv02.front.sepi 63.231.80.2 3 u 44 64 77 0.126 -0.771 0.764 2024-04-25T11:21:38.311 INFO:teuthology.orchestra.run.smithi106.stdout:+hv03.front.sepi 74.6.168.72 3 u 52 64 77 0.111 +0.092 0.598 2024-04-25T11:21:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:21:38.312 INFO:teuthology.orchestra.run.smithi134.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T11:21:38.312 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================== 2024-04-25T11:21:38.313 INFO:teuthology.orchestra.run.smithi134.stdout:+hv01.front.sepi 67.205.162.81 3 u 44 64 77 0.161 -2.252 0.743 2024-04-25T11:21:38.313 INFO:teuthology.orchestra.run.smithi134.stdout:*hv02.front.sepi 63.231.80.2 3 u 46 64 77 0.110 -0.915 0.690 2024-04-25T11:21:38.313 INFO:teuthology.orchestra.run.smithi134.stdout:+hv03.front.sepi 74.6.168.72 3 u 49 64 77 0.112 -0.140 0.702 2024-04-25T11:21:38.313 INFO:teuthology.orchestra.run.smithi134.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T11:21:38.314 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:21:38.351 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:21:38.352 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:21:38.409 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:21:38.465 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:21:38.474 INFO:teuthology.task.internal:Duration was 810.340538 seconds 2024-04-25T11:21:38.474 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:21:38.525 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:21:38.526 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:21:38.528 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:21:38.565 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:21:38.565 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T11:21:38.565 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:21:38.617 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T11:21:38.617 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:21:38.627 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:21:38.627 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:21:38.661 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:21:38.700 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:21:38.701 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:21:38.729 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:21:38.804 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:21:38.813 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:21:38.814 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:21:38.825 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:21:38.839 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:21:38.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:21:38.873 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:21:38.880 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-04-25T11:21:38.892 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-04-25T11:21:38.905 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:21:38.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:21:38.935 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:21:38.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:21:38.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:21:38.960 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:21:38.961 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/remote/smithi106 2024-04-25T11:21:38.961 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:21:39.000 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673120/remote/smithi134 2024-04-25T11:21:39.001 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:21:39.022 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:21:39.022 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:21:39.041 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:21:39.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:21:39.078 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:21:39.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:21:39.158 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:21:39.159 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:21:39.162 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:21:39.166 INFO:teuthology.orchestra.run.smithi106.stdout: 658621 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:21 /home/ubuntu/cephtest 2024-04-25T11:21:39.169 INFO:teuthology.orchestra.run.smithi134.stdout: 658625 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:21 /home/ubuntu/cephtest 2024-04-25T11:21:39.170 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:21:39.187 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:21:39.248 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-readahead} duration: 810.3405382633209 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T11:21:39.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:21:39.348 INFO:teuthology.run:pass